Purportedly AI-Generated False Citations in Arizona Probate Appeal Reportedly Led to Sanctions Against Self-Represented Appellant

Self-represented appellant Jeffrey Kotchka reportedly used purported AI tools to prepare an opening brief in a probate dispute involving his late mother's estate. The Arizona Court of Appeals found six of its eight citations deficient, including two nonexistent cases, and said Kotchka had made no effort to verify them. The court affirmed the appointment of his sister, Kim Dineen, as personal representative, ordering Kotchka to pay her attorney fees attributable to addressing the false citations.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The AI system generated false and misleading information in the form of fabricated legal citations, which were then submitted to a court of law.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: The litigant relied on generative AI to draft a legal brief in a court proceeding without verifying the citations, demonstrating unsafe use of AI in a critical situation.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by the AI system hallucinating non-existent case citations, which was an unexpected and unintentional outcome of using the tool to draft a brief.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system used was a general-purpose generative AI tool used for text generation, which falls under the category of AI systems that generate content and require transparency, but does not fall into high-risk categories like critical infrastructure or law enforcement.

AI system and alleged parties

• AI system: generative AI
• AI purpose: Writing Assistant; Technical Text Generation
• Behaviour type: Assistant
• Alleged developer: Large language model developers
• Alleged deployer: Pro se litigants, Litigants, Jeffrey Kotchka
• Alleged harmed parties: Pro se litigants, Litigants, Kim Dineen, Judicial integrity, Jeffrey Kotchka, Epistemic integrity, Arizona Court of Appeals

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Epistemic

Reported: The report explicitly describes epistemic harm in the form of fabricated legal citations submitted to a court of law.

Directly caused: The AI system generated six deficient citations, including two cases that 'simply do not exist', which were then submitted in a formal court filing.

Indirectly caused: The submission of fake citations required the appellate court and the opposing party to spend time and resources verifying and addressing non-existent legal precedents.

People affected

• Occurrences reported: 1
• People reportedly harmed: 2
• People reportedly exposed: 2

Potential causes

Management

• Poor Case Management: Litigant failed to manage the accuracy and quality of his own filing.

Technology

• Generative AI Hallucinations: AI generated fake, non-existent case citations in the legal brief.
• Lack of Built-in Verification: The AI tool did not verify the accuracy of the cited legal sources.

Data Inputs

• Inaccurate Training Data: Model associated wrong rules and facts with actual case names.

Human Factors

• Failure to Verify Citations: Litigant did not independently verify any of the AI-generated citations.
• Ignoring Court Guidelines: Litigant did not read the court's guide on using AI tools.
• No Professional Consultation: Litigant did not consult a legal professional before filing.

Process and Methods

• Lack of Verification Process: No process was used to cross-reference citations with public research tools.

Regulatory Environment

• Lack of Pre-filing AI Checks: No automated court system blocked the submission of fake citations.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides clear, unambiguous details about the court case, the specific citations that were fabricated, the parties involved, and the legal sanctions imposed. There are no conflicting accounts or major gaps in the narrative.

An Arizona litigant was sanctioned after using generative AI to draft a legal brief containing fabricated case citations. While representing a minor challenge for local state courts, the incident was resolved through standard judicial procedures and poses negligible threats to national security.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Individual litigant

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding the integrity of legal filings rather than an active national security crisis.
• Autonomy: Human-controlled. The AI was used as a tool to draft text, and the human litigant retained final control and responsibility for submitting the document.
• Novelty: Established threat. While a first for Arizona courts, similar incidents of AI-generated fake citations in legal briefs have occurred previously in other US jurisdictions.

Impact by dimension

• Physical security: Negligible. The incident is a local civil legal dispute with no physical impact, kinetic threats, or critical infrastructure disruption.
• Information security: Negligible. No intelligence compromise, systematic disinformation, or national security information warfare operations are indicated.
• Sovereignty: Minor. Minor disruption to state court proceedings caused by fraudulent AI-generated filings, which was easily managed and resolved through standard judicial sanctioning procedures.
• Economic security: Negligible. Minimal economic impact, limited strictly to private attorney fees and estate dispute assets, with no national economic or technological security implications.
• Societal stability: Negligible. No threat to societal stability, civil liberties, or public safety; the incident is a private family estate dispute.