State Farm Defense Counsel Acknowledged AI-Assisted Filings in Meni-Siliga v. A's Contractor, Inc. Contained Fabricated Legal Authorities

In Fa'alagilagi Meni-Siliga v. A's Contractor, Inc., et al., attorneys representing State Farm acknowledged that AI-assisted motions contained fabricated and otherwise inaccurate legal authorities. Attorney Jacquelene Robinson said she used the legal AI tool Irys and failed to verify some citations. Opposing counsel identified the errors, after which the firm apologized and filed corrected versions.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The AI system generated fabricated legal citations, quotes, and holdings, which constitutes false and misleading information that was subsequently submitted to a court.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: The attorney relied on the AI tool without verifying the citations, mistakenly believing it performed automatic cite-checking.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by the AI system Irys generating hallucinated, nonexistent legal citations, which was an unintended outcome of its use.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system is a generative AI tool used for drafting text, which falls under transparency obligations for chatbots and generative content.

AI system and alleged parties

• AI system: Irys
• AI purpose: Writing Assistant; Technical Text Generation
• Behaviour type: Assistant
• Alleged developer: Software developers, Iqidis, AI software developers, AI service providers
• Alleged deployer: Musick, Peeler & Garrett LLP, Legal professionals, Lawyers, Law firms, Jacquelene Robinson
• Alleged harmed parties: Parties to legal proceedings, Judicial integrity, Fa'alagilagi Meni-Siliga, Epistemic integrity

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Epistemic

Reported: Yes, the report explicitly describes epistemic harm in the form of fabricated legal citations, quotes, and case law submitted to a court.

Directly caused: The AI tool Irys generated seven nonexistent case citations across eight motions in limine, including fabricated quotes and holdings, which were then submitted in formal court filings.

Indirectly caused: The submission of fabricated legal precedents disrupted the legal proceedings, requiring opposing counsel to spend time verifying and debunking the non-existent cases.

People affected

• Occurrences reported: 1
• People reportedly harmed: 7
• People reportedly exposed: 7

Potential causes

Management

• Inadequate Counsel Oversight: State Farm did not monitor external counsel's AI policies and tools.
• Lack of Supervisory Control: Lead attorney did not review the second-chair's reliance on AI.

Technology

• AI Hallucinations in Irys: The AI tool Irys generated fabricated case citations and quotes.
• Lack of Westlaw Integration: Irys did not connect to Westlaw to perform internal cite checks.

Human Factors

• Mistaken Belief in Tool Checks: Attorney incorrectly assumed Irys automatically verified citations.
• Neglecting Manual Cite Checks: Attorney failed to manually verify the cited authorities before filing.
• Lack of Lead Counsel Awareness: Lead counsel was unaware that AI was used to prepare the filings.

Process and Methods

• Inadequate Filing Review: The firm lacked a formal process to verify AI-generated citations.
• Lack of Clear AI Policy: The firm had not implemented specific guidelines for generative AI use.

Regulatory Environment

• Lack of External AI Audits: No regulatory body or court mechanism pre-screened AI legal filings.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, consistent, and detailed accounts of the attorney using the AI tool Irys, the specific fabricated citations generated, and the subsequent court filings and apologies. There is no significant disagreement on the facts of the AI's failure.

An attorney in a civil lawsuit unintentionally submitted AI-generated fabricated legal citations to a California court. The incident represents a professional negligence and AI hallucination issue with negligible national security implications.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. This represents an ongoing concern regarding the professional use of generative AI rather than an imminent national security crisis.
• Autonomy: Human-controlled. The AI tool acted as a writing assistant to help draft motions, with the attorney retaining final control and responsibility for the filing.
• Novelty: Established threat. AI hallucinations and the submission of fabricated citations in court filings are established phenomena with multiple prior documented occurrences.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or human safety were impacted by this civil legal dispute.
• Information security: Negligible. The incident involves unintentional AI hallucinations in a standard civil lawsuit, with no indication of state-sponsored information operations or intelligence compromise.
• Sovereignty: Negligible. The incident caused minor, temporary disruptions to a local civil court proceeding but does not threaten state authority, constitutional processes, or core government functions.
• Economic security: Negligible. The economic impact is limited to minor unquantified administrative and legal costs for the involved private parties and local court, posing no threat to national economic stability.
• Societal stability: Negligible. There is no threat to societal stability, civil liberties, or population-scale safety. The impact is confined to a private legal dispute.