Citation Errors in Concord Music v. Anthropic Attributed to Claude AI Use by Defense Counsel

In a legal filing in Universal Music Group et al. v. Anthropic, lawyers for Anthropic acknowledged that expert witness testimony submitted in the case contained erroneous citations generated by the company's Claude AI system. The filing stated that the inaccuracies, which included incorrect article titles and author names, were not caught during manual review. Anthropic characterized the issue as an honest mistake and apologized in court.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The AI systems generated fabricated legal citations and case details (hallucinations) which were then submitted to courts as real authorities, misleading legal professionals and judges.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Lawyers over-relied on AI systems to perform critical legal research without verifying the accuracy of the generated citations.
• 7.3 Lack of capability or robustness: The AI systems failed to perform reliably, generating plausible-sounding but completely fabricated legal citations.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by AI systems generating hallucinated, non-existent citations during active use by legal professionals.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The systems involved are chatbots and generative AI assistants used for text generation. According to the definitions, 'Chatbots' fall under Limited Risk, requiring transparency so users know they are interacting with AI.

AI system and alleged parties

• AI system: Claude, CoCounsel, Google Gemini, Westlaw Precision (Anthropic, Google, Thomson Reuters)
• AI purpose: Writing Assistant; Technical Text Generation
• Behaviour type: Assistant
• Alleged developer: Anthropic
• Alleged deployer: Anthropic
• Alleged harmed parties: Sarang V. Damle, Sara E. Sampoli, Rachel S. Horn, Latham & Watkins LLP, Judicial process, Judicial integrity, Joseph R. Wetzel, Ivana Dukanovic, Brittany N. Lovejoy, Anthropic, Andrew M. Gass, Allison L. Stillman

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Minor, indirect Minor
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: Yes, the report explicitly describes financial losses in the form of court-ordered sanctions.

Directly caused: In the Pratt & Associates and K&L Gates case, the law firms were ordered to jointly pay $31,100 in the defendant's legal fees as a sanction.

Indirectly caused: The firms also suffered indirect financial harm from lost discovery relief and potential reputational damage affecting future business.

Inferred additional harm: Other lawyers involved in similar incidents, such as Jisuh Kim in Toronto, likely faced financial costs from contempt proceedings or lost billable hours, estimated at an additional $5,000 to $10,000.

Epistemic

Reported: Yes, the reports describe extensive epistemic harm through the fabrication of legal citations and case law by AI systems.

Directly caused: AI systems fabricated non-existent legal citations, case names, and journal article authors (e.g., Claude hallucinating a citation with an inaccurate title and authors, and other tools generating completely non-existent cases).

Indirectly caused: Lawyers incorporated these fabrications into official court briefs, misleading judges who then had to spend time researching non-existent authorities.

Inferred additional harm: The proliferation of AI-generated legal hallucinations increases the noise and verification burden within the legal information ecosystem, potentially leading to corrupted judicial precedents if undetected.

People affected

• Occurrences reported: 3
• People reportedly harmed: 6
• People reportedly exposed: 15

Potential causes

Management

• Lack of AI Use Policies: Law firms failed to establish strict protocols for verifying AI-generated work.

Technology

• AI Hallucination: Large language models generate plausible-sounding but fabricated information.
• Lack of Fact-Checking Features: Chatbots lack built-in real-time validation of generated references.

Data Inputs

• Training Data Limitations: Models are trained on text patterns rather than verified databases.

Human Factors

• Overreliance on AI Outputs: Lawyers assumed the chatbot outputs were accurate without verification.
• Lack of AI Literacy: Users did not understand that LLMs routinely fabricate citations.

Process and Methods

• Inadequate Citation Verification: Manual review processes failed to check the validity of generated citations.
• Poor Internal Communication: Lawyers shared drafts without disclosing that AI was used to generate them.

Regulatory Environment

• Unclear Professional Guidelines: Lack of clear, enforced standards on AI use in legal document preparation.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, consistent, and detailed accounts of multiple legal cases where AI-generated hallucinations were submitted to courts. The specific AI tools, law firms, judges, and consequences (such as the $31,100 sanction) are explicitly documented across multiple reliable sources.
• Ambiguities identified: The exact technical prompts used by the lawyers to generate the citations are not fully detailed.
• Alternative interpretations: None. The facts clearly point to AI hallucination combined with a lack of human verification.

Multiple lawyers in the US and Canada submitted court filings containing AI-hallucinated citations, leading to judicial sanctions. While these incidents highlight risks of professional overreliance on AI and temporary disruptions to court proceedings, they pose negligible threats to national security and are managed through standard judicial processes.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: United States, Canada
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing professional capability and verification concern rather than an active national security crisis.
• Autonomy: Human-supervised. The AI systems generated the content autonomously, but human legal professionals maintained final oversight and authority over the submissions.
• Novelty: Established threat. AI hallucinations and the submission of fake citations in court have occurred in multiple prior well-publicized instances.

Impact by dimension

• Physical security: Negligible. No physical systems, infrastructure, or human safety risks were involved in this legal research incident.
• Information security: Negligible. The incident involves unintentional legal research errors rather than systematic information warfare, intelligence compromise, or hostile state operations.
• Sovereignty: Minor. While the integrity of judicial proceedings was temporarily challenged by fake citations, the issue was successfully managed within standard court procedures and sanctions, posing no threat to core state authority.
• Economic security: Negligible. The financial impact was limited to minor court-ordered sanctions on specific law firms, with no broader threat to national economic stability or strategic technological security.
• Societal stability: Negligible. The incident did not involve mass surveillance, algorithmic oppression, or threats to civil liberties and population safety.