Lawyer in Amarsingh v. Frontier Airlines Sanctioned Over Court-Identified ChatGPT Citation Errors

In Amarsingh v. Frontier Airlines, the Tenth Circuit sanctioned attorney Kusmin L. Amarsingh after finding that her appellate brief included fabricated case citations and misattributed legal authority generated by ChatGPT and not verified before filing. The court ordered a $1,000 payment to Frontier and referred the matter to Maryland disciplinary authorities.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The AI system generated fabricated legal citations and quotes (false information) which misled the attorney into submitting them to a federal court, resulting in financial sanctions.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: The attorney overrelied on ChatGPT for legal research and failed to perform the fundamental duty of verifying the accuracy of the generated citations.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was initiated by the AI system generating fabricated legal citations and quotes, which is an unintentional hallucination outcome of the model's generation process.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3. Limited Risk: The system involved is ChatGPT, a general-purpose chatbot and generative AI tool. The report describes the attorney's 'reliance on generative AI tools, specifically ChatGPT, to assist in legal research and drafting'.

AI system and alleged parties

• AI system: ChatGPT (OpenAI)
• AI purpose: Writing Assistant; Technical Text Generation
• Behaviour type: Assistant
• Alleged developer: OpenAI, Large language model developers, Chatbot developers, AI research tool developers
• Alleged deployer: Pro se litigants, Lawyers, Kusmin L. Amarsingh
• Alleged harmed parties: United States Court of Appeals for the Tenth Circuit, Judicial integrity, Frontier Airlines, Epistemic integrity

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Epistemic

Reported: Yes, the report explicitly describes epistemic harm in the form of fabricated legal citations and quotes generated by ChatGPT.

Directly caused: ChatGPT generated seven non-existent court cases and a fabricated quote, which were then submitted in a formal legal filing to the 10th Circuit Court of Appeals.

Indirectly caused: The submission of fake citations wasted judicial resources and damaged the credibility of the legal system.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 1

Potential causes

Management

• Neglect of Professional Duty: The lawyer failed to manage and verify her own legal work product.

Technology

• Generative AI Hallucination: ChatGPT generated plausible-sounding but completely fictitious legal case names.
• Non-Existent Case Quotation: The AI tool fabricated quotes and attributed them to non-existent cases.
• Plausible Output Formatting: AI output matched standard legal citation styles, masking its inaccuracy.

Data Inputs

• Lack of Real-Time Case Validation: The AI model lacked integration with live, verified legal databases.

Human Factors

• Overreliance on GenAI Tools: The lawyer assumed ChatGPT output was accurate without independent verification.
• Failure to Cross-Reference: The attorney failed to check the generated citations against actual case texts.
• Misunderstanding of AI Sophistication: The attorney believed the tool's sophistication guaranteed factual accuracy.

Process and Methods

• Inadequate Legal Brief Review: The process of drafting lacked a mandatory verification step for citations.
• Careless Use of AI in Research: The drafting process relied solely on AI output for critical legal precedents.

Regulatory Environment

• Evolving Professional Rules: Rules warning against improper AI use were only recently being amended.
• Lack of Pre-Filing Warnings: Courts were only beginning to explicitly caution lawyers about AI pitfalls.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides clear, unambiguous details about the incident, including the specific court order, the attorney's admission, the exact number of fake citations, and the financial penalty. There are no conflicting accounts.

An attorney was sanctioned $1,000 by a U.S. federal appeals court for submitting a brief containing fabricated citations generated by ChatGPT. The incident represents a case of professional negligence and AI overreliance with negligible implications for national security.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Individual

Threat characteristics

• Imminence: Long-term. The incident is a completed event that was resolved through court sanctions, presenting no immediate or near-term national security crisis.
• Autonomy: Human-controlled. The AI system acted as a drafting assistant, with the human attorney retaining final authority and responsibility for filing the document.
• Novelty: Established threat. Generative AI hallucinations in legal filings are a known and previously documented phenomenon, representing an established threat pattern.

Impact by dimension

• Physical security: Negligible. The incident involved a legal filing and had no impact on physical systems, critical infrastructure, or human safety.
• Information security: Negligible. There was no compromise of intelligence capabilities, classified information, or systematic state-sponsored information warfare.
• Sovereignty: Negligible. While the incident wasted federal judicial resources, it was successfully resolved through standard court sanctioning procedures and poses no threat to state authority or core government operations.
• Economic security: Negligible. The incident resulted in a minor individual financial sanction of $1,000 and has no impact on strategic industries, national economic stability, or technological competitiveness.
• Societal stability: Negligible. The incident was a localized legal dispute with no implications for mass surveillance, civil liberties, or population-scale social cohesion.