The complete rollout comes weeks after the TritonGPT, was upgraded to Llama 3, a large language model (LLM) that offers better reasoning abilities, code generation and instruction-following capabilities.
There’s been some surprises, as well as notable benefits. According to UCSD Chief Information Officer Vince Kellen, the cost savings compared to using external AI systems have been significant.
“The whole purpose of this exercise is to keep the cost frighteningly low, which we’ve been able to do,” said Kellen, adding that powering the AI suite costs the university about one-tenth of what it would for an external subscription service.
ROLLING OUT AI IN HIGHER ED
The initial 400 users were selected by nomination, followed by a full rollout — first to financial staff, and eventually to faculty members. Usage patterns vary across different employee groups, with 20 percent of users accounting for 80 percent of queries, as noted by Kellen.
YouTube
“Faculty are people too, and I think they’re very much like most people and there’s a spectrum of, ‘This is so cool,’ to, ‘This is a work of the devil,’” said Kellen.
If that’s the case, the devil is working in the details — the tedious details. The university’s AI assistants are taking over tasks that most staff prefer not to do.
“It’s sort of like picking off the low-hanging fruit,” said Kellen. “We’re not doing super high-risk uses — like giving students advice on which major to pick, that’s got a lot of risk to it, and a lot of uncertainty. Instead, we’re focusing on the myriad of mundane things we do in the university, whether it be instruction or administration.”
WHAT CAN AI DO AT A UNIVERSITY?
The TritonGPT suite currently offers a number of assistants, each designed for specific tasks, with more in development to expand its capabilities.
According to Kellen, one of the first things many users do is ask the AI assistant about themselves (a practice he jokingly referred to as "glory searching").
Other common uses are summarizing or rewriting text, brainstorming and crafting job descriptions with AI.
AI-POWERED JOB DESCRIPTIONS
The Job Description Writer is an AI assistant that allows staff to input some data and receive an AI-created description that has been reviewed to remove unintended bias or wording that may discourage certain groups from applying.
The Job Description Writer has been particularly well-received. Kellen notes that it has significantly reduced the time managers spend on this task, and human resources has reported higher quality applicants since its launch.
“A lot of managers spend too much time writing job descriptions, and when we released this it turned it into a five-minute thing, rather than a half-hour ordeal,” said Kellen.
AI-ASSISTED GRANT MANAGING
One of the newer assistants, the Fund Manager Coach, is an AI that provides personalized advice for the employees who oversee grants and the associated finances.
It's still too early to gauge its effectiveness.
“People are very excited about it, no question, but satisfaction, it’s a little too early to get in on that one,” said Kellen.
SERVE AS AN AI-POWERED CAMPUS EXPERT
UCSD Assistant helps staff and faculty navigate university policies, processes and documentation. The General AI assistant, on the other hand, is designed for broader tasks like summarizing documents, generating ideas and writing emails or reports.
According to the project website, myriad data was used to train the AI assistants, including the admissions website, the business analytics hub, university policies and a ServiceNow knowledge base.
Kellen reports that overall user feedback has been overwhelmingly positive, likely due in part to the voluntary nature of adoption.
“Most new technologies do not receive positive reviews when they come out in the enterprise space,” he said. “If we bring out a new payroll system, everybody hates it for the first year. But here, because it’s more voluntary, the feedback has been really good.”
CUTTING AI COSTS IN THE PUBLIC SECTOR
TritonGPT wasn't UCSD's first foray into AI. The university leveraged its experience monitoring GPU usage from past machine learning projects to inform data planning for TritonGPT. They tested out the AI, saw it worked well, then bought powerful computers to run it.
According to Kellen, they’re now at a point where they have all the computer power they need, with plenty of power to handle more users in the future. They're watching how it's being used every day and have tools to make it run faster.
“If you go to Microsoft and OpenAI, they’re going to want to charge you for enterprise class stuff for $30 per user per month,” said Kellen. “That seems to be a common price, maybe $20, or $40. But around that $30 per user. Right now we’re engineered at one-tenth of that price.”
The university is working with Danswer.ai, a new company founded by UCSD alumni, to produce the user interface.
In the future UCSD will partner with Protopia to enhance data protection measures within TritonGPT. This includes encrypting information entered into chat windows to prevent any accidental disclosure of personal information.
Grades and other personal information about an individual is off-limits at the moment. The AI system is designed to protect private information; most of the data it uses doesn't include personal details.
In the future, they plan to connect TritonGPT to databases for tasks such as grant management, but with strict safeguards to ensure only general, non-personal information is accessible.
NAVIGATING RESPONSIBLE AI
While TritonGPT has proven valuable for staff and faculty, a looming question remains: What about the students?
For now, the tool is not available to students. Kellen explained that the university wants to prioritize safety and accuracy before expanding access to a younger audience.
“We want to get further along on all the safety mechanisms because students don’t always listen to adults,” said Kellen. “We want to make sure we get all of the safety really ironed out before we get there.”
Additionally, it’s not being used for tasks like matching students with educational paths. Kellen emphasized the importance of addressing potential issues with "hallucination" and accuracy in AI-generated responses.
“If the data are skewed that the language model is trained on, then the responses can be skewed,” he said. “Obviously, nothing is going to be perfect in this realm. And that’s not our goal, perfection. Our goal is just to get as high as we can possibly get on the quality and the safety.”
WHAT’S NEXT?
The potential applications for TritonGPT continue to expand, with new use cases emerging as the technology matures.
Recent successful pilot projects have demonstrated TritonGPT's potential for tasks like streamlining work-study reporting and summarizing lectures.
However, one big goal for the future is to increase adoption. Kellen acknowledges that experts may be less inclined to embrace new tools, especially if they don't immediately see the value.
Kellen's hope is to increase engagement by demonstrating the tool's usefulness. That may require a more clear cut path for use and training.
“You have to marry technology to a very specific task,” said Kellen. “And that’s part of user training.”
