Key takeaways

  • A New Mexico attorney was held in contempt after filing an appeal citing fake witness testimony made up by ChatGPT.
  • The attorney was fined $5,000, barred from appearing before the court, and referred for formal disciplinary action.
  • Judges held that relying on unverified LLM outputs constitutes an unacceptable failure of professional responsibility.

What happened

The New Mexico Supreme Court found veteran criminal defense attorney Stephen Aarons in direct contempt of court after he submitted an appellate brief containing fabricated witness statements and distorted case citations produced by an artificial intelligence model.

Aarons, who was hired to appeal a life sentence in a murder conviction, admitted that he fed computerized trial transcripts into ChatGPT, running on OpenAI's o3 model, and failed to cross-check the resulting text against the official trial record before filing.

The resulting court filing included quotes from individuals who were never called to the witness stand during the original trial, misattributed statements from actual witnesses, and misstated legal precedent. Aarons acknowledged that he did not examine the factual validity or legal citations in the draft, nor did he alert his client to the presence of unverified AI-generated content.

In response to the errors, the state's highest court vacated the existing briefs, assigned a public defender to take over the appeal, and delayed proceedings into the 2026-2027 court term.

During hearings, the justices sharply rebuked Aarons, levying a $5,000 fine payable to the state bar's client protection fund and indefinitely barring him from practicing before the state supreme court pending formal disciplinary proceedings. The court emphasized that the professional duty of candor and verification remains entirely with the human signatory, irrespective of whether the text was produced by software or human staff.

Why it matters

This case highlights a critical boundary in enterprise and professional AI adoption: reliance on advanced frontier models does not dilute human accountability. While early legal controversies involved large language models inventing non-existent case citations, this incident demonstrates that models can subtly hallucinate witness names, factual timelines, and courtroom statements while summarizing authentic source documents.

Enterprise users often assume that summarization tasks are inherently safer than open-ended generation, but complex documents like legal transcripts remain susceptible to factual distortion and invented details.

Furthermore, the court firmly rejected the defense of technological ignorance, arguing that the public discourse surrounding model hallucinations is now sufficiently established that professionals have a proactive duty to understand AI limitations. For builders of legal tech and enterprise agentic workflows, the ruling underscores the urgent necessity for strict provenance tracking, automated citation grounding, and mandatory human-in-the-loop validation rather than relying purely on model self-evaluation.

What to watch

Moving forward, expect judicial bodies, corporate compliance departments, and professional licensing boards to accelerate the formal rollout of binding generative AI disclosure rules and audit protocols. Developers building document summarization, retrieval-augmented generation (RAG), and specialized agentic workflows must prioritize verifiable source attribution and grounding features to guarantee that hallucinated assertions cannot contaminate mission-critical outputs.

Legal technology providers face growing commercial and regulatory pressure to prove their platforms actively guard against subtle synthesis errors, as client liability concerns will increasingly dictate enterprise procurement standards. As courts establish zero-tolerance standards for unverified AI materials, software vendors that offer transparent verification chains and automated discrepancy detection will secure a distinct competitive advantage over general-purpose chat interfaces.