Reported False Legal Citations in South African Case Mavundla v. MEC Generated by Purported AI Tool

In Mavundla v. MEC: Department of Co-Operative Government and Traditional Affairs KwaZulu-Natal, the law firm Surendra Singh and Associates, representing South African politician Philani Godfrey Mavundla, reportedly submitted false legal citations, apparently generated by AI, in filings at the Pietermaritzburg High Court. The court reportedly found that many of the cited cases did not exist in any databases, and the firm's legal team and article clerk allegedly failed to verify the references.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The AI system generated completely fictitious legal citations and case details, which were then submitted to the court as factual authorities.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: The legal practitioners over-relied on the AI-generated research and failed to perform the necessary professional verification before submitting the documents to court.
• 7.3 Lack of capability or robustness: The AI system exhibited a lack of robustness by hallucinating realistic-looking but entirely non-existent legal citations when queried for South African case law.

Causal factors

• Entity: Human
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by human legal practitioners failing to verify AI-generated outputs before submitting them to court, which was an unintentional oversight during post-deployment use of the technology.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The AI systems used were general-purpose chatbots and text generation tools (ChatGPT and a specialized legal AI assistant) which fall under transparency obligations for interactive AI and synthetic content generation.

AI system and alleged parties

• AI system: ChatGPT, unspecified AI platform (OpenAI)
• AI purpose: Writing Assistant; Question Answering
• Behaviour type: Assistant
• Alleged developer: Unnamed large language model developers, Unnamed generative AI companies
• Alleged deployer: Surendra Singh and Associates
• Alleged harmed parties: Surendra Singh and Associates, Philani Godfrey Mavundla

Harm severity

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Epistemic

Reported: The report explicitly describes epistemic harm through the generation and submission of fabricated legal citations and case law.

Directly caused: The AI system generated entirely fictitious case citations (such as 'Pieterse v The Public Protector') which were presented to the court as real legal authorities.

Indirectly caused: The judge's independent search using ChatGPT resulted in the AI falsely confirming the existence of a non-existent case, further spreading fabricated legal information.

Inferred additional harm: N/A

People affected

• Occurrences reported: 2
• People reportedly harmed: 6
• People reportedly exposed: 6

Potential causes

Management

• Failure of Staff Supervision: Senior professionals failed to properly supervise candidate attorneys' work.
• Inadequate Quality Control: Firms lacked rigorous internal review mechanisms for court submissions.
• Resource Constraints: Firms lacked access to or refused to pay for official law databases.

Technology

• AI Hallucinations: AI tools generate coherent but entirely fictitious legal case citations.
• Unreliable AI Confirmatory Checks: AI chatbots falsely confirm the existence of non-existent cases.

Data Inputs

• Unverified Reference Databases: AI models trained on legal data still produce fabricated citations.

Human Factors

• Lack of Professional Vigilance: Practitioners failed to verify AI outputs against authoritative sources.
• Dishonesty Regarding AI Use: Staff denied using AI tools when questioned about fictitious cases.
• Overreliance on AI Tools: Assuming AI-generated research is accurate without critical evaluation.

Process and Methods

• Inadequate Verification Protocols: Lack of systematic cross-checking of citations before court submission.
• Time Pressure and Urgency: Severe time constraints and overbooking led to rushed drafting processes.
• Deficient AI Risk Training: Lack of training on AI limitations, risks, and ethical implications.

Regulatory Environment

• Evolving Legal Standards: Absence of clear, preemptive guidelines on AI usage in court filings.
• Mandatory Judicial Referrals: Code of Judicial Conduct obliges judges to report professional misconduct.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide highly detailed, consistent accounts of both court cases, including specific case names, the names of the legal practitioners involved, the judges' rulings, and the exact nature of the AI hallucinations. The role of AI is explicitly confirmed by the practitioners in the Northbound case and strongly established via judicial investigation in the Mavundla case.
• Ambiguities identified: The exact AI platform used in the Mavundla case is not definitively named, though the judge used ChatGPT to verify it and the candidate attorney denied using AI.
• Alternative interpretations: None. The facts clearly point to professional negligence involving AI-generated hallucinations.

Legal practitioners in South Africa submitted AI-generated fictitious case citations in court. While this temporarily disrupted judicial proceedings and highlighted risks of overreliance on AI in legal research, the incident represents negligible national security risk and was handled through standard judicial disciplinary processes.

• Overall national security impact: Minor
• Response level: Moderate
• Scope: Single nation
• Primary target: South Africa
• Alleged perpetrator: Legal practitioners

Threat characteristics

• Imminence: Long-term. This represents an ongoing professional standards and technological integration issue rather than an active national security crisis.
• Autonomy: Human-controlled. The AI tool was used as a writing assistant, and humans retained final control over the submission of the documents, despite failing to verify the output.
• Novelty: Established threat. AI hallucinations in legal filings have been documented globally in prior incidents, making this an established threat pattern.

Impact by dimension

• Physical security: Negligible. No threats to physical systems, critical infrastructure, or human safety were reported in connection with this incident.
• Information security: Negligible. The incident involved localized legal document generation and did not compromise intelligence capabilities or represent systematic information warfare.
• Sovereignty: Minor. Fictitious legal citations submitted to courts temporarily disrupted judicial proceedings, representing a minor impact on core government legal functions that was managed via standard procedures.
• Economic security: Negligible. The economic impact was limited to punitive cost orders for individual law firms, with no threat to national economic or technological security.
• Societal stability: Negligible. There was no threat to large-scale social cohesion, civil liberties, or population safety, despite localized concerns regarding judicial integrity.