The motion included false information known in artificial intelligence circles as “hallucinations,” meaning that it was invented by the AI software asked to write the material, Wilson said. It was filed in connection with the case of Kalen Turner, who was accused of five felony and two misdemeanor drug counts, he said.
“A prosecutor recently used artificial intelligence in preparing a filing, which resulted in an inaccurate citation,” Wilson said in the statement to The Sacramento Bee. “Once the error was discovered, the filing was immediately withdrawn.”
Wilson’s also said his deputies had filed briefs with inaccurate legal references in two other cases, but that they were caused by human error, not AI.
The situation is the latest example of the potential pitfalls connected with the growing use of AI. In fields such as law, errors in AI-generated briefs could impact the freedom of a person accused of a crime. In health care, AI analysis of medical necessity has resulted in the denial of some types of care. In April, A 16-year-old Rancho Santa Margarita boy killed himself after discussing suicidal thoughts with an AI chatbot, prompting a new California law aimed at protecting vulnerable users.
“While artificial intelligence can be a useful research tool, it remains an evolving technology with limitations — including the potential to generate ‘hallucinated’ citations,” Wilson said. “We are actively learning the fluid dynamics of AI-assisted legal work and its possible pitfalls.”
All three of the Nevada County cases are the subject of a petition filed late last month with the California Supreme Court by a defendant in one of them.
Lawyers for Kyle Kjoller, who faced numerous firearms-related charges, said that the errors in each case has the hallmarks of AI hallucinations. They are requesting an investigation and asking for sanctions against the DA’s office.
The Supreme Court has not yet said whether it will consider Kjoller’s petition, which was denied last month by a state appeals court.
“The Nevada County District Attorney’s Office has, in at least three criminal cases in recent weeks, filed briefing citing to fabricated (legal) authority,” wrote lawyers for the nonprofit Civil Rights Corps, which is representing Kjoller along with a Nevada County public defender.
In his petition, Kjoller, who was ultimately convicted on the firearms charges, alleges that all three cases included examples of AI use gone wrong. The petition includes examples of six cases cited in a brief filed in connection with Kjoller’s bail hearing that his lawyers say appear to be hallucinations.
“Case does not exist,” they wrote in a chart detailing the citations. “Reporter citation does not exist. Does not accurately describe.”
The brief also mischaracterized a provision of the California state Constitution, they said.
Worries about fictions created by artificial intelligence used to prepare legal documents have plagued the legal community for the past few years, as the public’s infatuation with the generative technology has grown. When used in legal cases, AI hallucinates between 17% adn 82% of the time, Kjoller’s petition suggested.
Using incorrect or false case references in legal documents has real implications for defendants, because judges rely on these references to help form their rulings, Kjoller’s lawyers wrote.
“Citing to fabricated authority runs the very real risk that a court may rely on a prosecutor’s misrepresentation in its ruling,” attorneys Kjoller’s lawyers wrote in their petition. “In criminal cases, this can have horrific, life-shattering results.”
The petition contends that Wilson’s office did not acknowledge the use of AI in the Turner case, despite pointed questions on the topic from the judge. It also contends that Wilson’s office did not correct its errors in the other two cases.
Kjoller’s lawyers also claim that Wilson’s chief deputy, Lydia Stuart, threatened them with her own sanctions motion, which she denies.
In an email included with the petition, Stuart does point to a California law barring the filing of frivolous requests for sanctions against opposing attorneys. But she says this was in response to a question and was not a threat.
“While we respect and expect zealous advocacy ... the assertion that our filing contained ‘hallucinated holdings’ and was ‘so obliquely and misleading as to suggest the entire argument was written by generative AI’, itself strikes as a misleading characterization to the court,” she wrote in the email.
“Here in Nevada County,” she added, “we strive to maintain respectful adversarial practice with our professional colleagues and refrain from exploiting inadvertent errors that arise from the demands of our profession and limited resources.”
Wilson said his office took immediate steps to address the AI problem, speaking directly to the attorney involved and reminding his staff to double-check and confirm any information that it generates.
“All of the attorneys in the office were reminded to verify all legal citations independently and not rely on AI-generated material without confirmation from reliable sources,” he said.
But Wilson insisted that errors in Kjoller’s case and that of Taylor McGrath, who was accused of felony child endangerment along with drug charges, did not involve artificial intelligence.
Just because the Turner case involved AI, “it cannot now be assumed that every citation error stems from the use of artificial intelligence,” he said.
“Prosecutors work diligently and in good faith under heavy caseloads and time constraints, with a deep sense of responsibility to the law, the Court, and the community we serve,” Wilson said. “At no time was there any intent to mislead the court, and any characterization to the contrary misstates the facts.”
© 2025 The Sacramento Bee. Distributed by Tribune Content Agency, LLC.
