Law Firm Shapiro Arato Bach Reportedly Filed Fictitious Case Citation and Misattributed Quotations Using Harvey's Legal AI Platform in New York Federal Court

Law firm Shapiro Arato Bach reportedly acknowledged that unchecked use of Harvey, a legal AI platform, produced a fictitious case citation and misattributed quotations in a brief filed in federal court in New York. The court required an explanation and disclosure of the AI tool. The judge declined sanctions, and the firm apologized and said it was adding safeguards.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The AI tool generated fabricated legal citations and false quotations (hallucinations) which were then submitted to a federal court, presenting false information as fact.

Additional risk subdomains

• 7.3 Lack of capability or robustness: The legal AI tool exhibited a lack of robustness by hallucinating non-existent case citations and misattributing legal principles.
• 5.1 Overreliance and unsafe use: The drafting lawyer overrelied on the AI tool's output, failing to perform the standard professional verification of the generated citations.

Causal factors

• Entity: Human
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by a human lawyer failing to verify AI-generated outputs before filing them in court, which was an unintentional oversight contrary to firm policy.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system is a generative AI tool used for drafting text (similar to a chatbot or content generator) which requires transparency and human verification, but does not fall under high-risk categories like critical infrastructure or law enforcement.

AI system and alleged parties

• AI system: None named
• AI purpose: Writing Assistant; Technical Text Generation
• Behaviour type: Assistant
• Alleged developer: Large language model developers, Harvey, AI-assisted legal drafting system developers
• Alleged deployer: Shapiro Arato Bach LLP, Lawyers, Law firms
• Alleged harmed parties: United States District Court for the Southern District of New York, Judicial integrity, Courts

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 1
• People reportedly exposed: 2

Potential causes

Management

• Inadequate Training Enforcement: Mandatory AI ethics training failed to prevent individual compliance errors.
• Prioritizing Margin over Compliance: Corporate pushed AI pricing recommendations to optimize top-line revenue.
• Lack of Mentorship and Oversight: Insufficient supervision of junior lawyers using generative AI tools.

Technology

• Generative AI Hallucination: The Harvey AI tool generated fake citations and quotes that did not exist.
• Opaque Pricing Recommendations: AI-driven pricing engine created confusing and wide variations in food prices.
• Complex Algorithmic Rules: AI engine parameters and corporate targets created conflicting pricing signals.

Data Inputs

• Competitor Price Scraping: Pulling online menu data from competitors introduces legal and antitrust risks.
• Granular Transaction Data Analysis: Analyzing millions of daily transactions led to localized price discrimination.

Human Factors

• Failure to Verify AI Outputs: The responsible lawyer did not check the AI-generated case citations.
• Distraction and Personal Bereavement: The drafting lawyer suffered personal loss, impacting her focus and diligence.
• Overreliance on Well-Known Principles: Partner assumed citations were correct because principles were uncontroversial.

Process and Methods

• Violation of Firm AI Policy: The drafting attorney acted contrary to the firm's AI validation policies.
• Lack of Independent Draft Verification: No secondary process caught the hallucinated citations before court filing.
• Aggressive Franchisee Pressure: Corporate enforcement systems penalized franchisees who deviated from AI pricing.

Regulatory Environment

• Antitrust and Collusion Scrutiny: Regulators like the FTC scrutinize algorithms that coordinate competitor prices.
• Increasing Judicial Sanctions: Courts are increasingly weighing sanctions for unchecked AI legal submissions.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide highly detailed, first-hand legal declarations and court details explicitly outlining how the AI was used, what errors were generated, and the organizational context of the failure.

A private law firm submitted a court filing in a civil copyright case containing fabricated legal citations generated by an AI tool. While this represents a failure of professional conduct and highlights risks of unchecked AI use in the legal system, it was an isolated, unintentional error with negligible implications for national security.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The incident has been resolved by the court and represents an ongoing professional and regulatory challenge rather than an active national security crisis.
• Autonomy: Human-controlled. The AI was used as a writing assistant to generate draft text, which was intended to be reviewed and filed by human lawyers.
• Novelty: Established threat. AI hallucinations and lawyers filing unverified AI-generated briefs with fabricated citations have occurred in multiple prior cases.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or human safety elements were threatened or impacted by this legal filing error.
• Information security: Negligible. The incident involved unintentional AI hallucinations in a routine civil copyright lawsuit, not a state-sponsored disinformation or intelligence compromise operation.
• Sovereignty: Minor. The submission of fabricated citations briefly affected a federal court proceeding, but it was quickly resolved by the judge without disrupting core government operations or constitutional processes.
• Economic security: Negligible. The incident represents an isolated professional negligence issue within a private law firm and does not threaten strategic industries, national economic stability, or critical supply chains.
• Societal stability: Negligible. There was no threat to public safety, civil liberties, or societal cohesion; the impact was limited to the parties involved in a private legal dispute.