South African Legal Team Reportedly Relied on Unverified ChatGPT Case Law in Johannesburg Body Corporate Defamation Matter

In a defamation case at the Johannesburg Regional Court, Rodrigues Blignaut Attorneys, representing plaintiff Michelle Parker, reportedly relied on purportedly non-existent legal judgments generated by ChatGPT to help argue their case. Magistrate Arvin Chaitram reportedly found the case names and citations were fictitious, causing a two-month delay. The court issued a punitive costs order and rebuked the plaintiff's legal team for uncritically accepting AI-generated research.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The incident involves ChatGPT generating fabricated legal citations, which falls under false or misleading information.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: The lawyers overrelied on the chatbot's outputs for critical legal research without verifying them.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by ChatGPT generating fictitious legal citations (AI entity) unintentionally during post-deployment use.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: ChatGPT is a chatbot, which is explicitly classified as a limited risk system requiring transparency under the EU AI Act.

AI system and alleged parties

• AI system: ChatGPT (OpenAI)
• AI purpose: Writing Assistant; Question Answering
• Behaviour type: Assistant
• Alleged developer: OpenAI
• Alleged deployer: Rodrigues Blignaut Attorneys, Jurie Hayes, Chantal Rodrigues
• Alleged harmed parties: Rodrigues Blignaut Attorneys, Michelle Parker, Legal integrity, Jurie Hayes, Judicial integrity, Epistemic integrity, Chantal Rodrigues

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Psychological

Reported: Yes

Indirectly caused: The plaintiff's attorneys suffered significant professional embarrassment and reputational damage after their careless use of ChatGPT was exposed in a published court judgment.

Epistemic

Reported: Yes

Directly caused: ChatGPT generated entirely fictitious legal citations, case names, and judicial decisions, presenting them as factual legal precedents.

Indirectly caused: The fabrication of legal precedents misled the opposing legal team and caused a two-month delay in court proceedings as multiple lawyers and librarians spent hours searching for non-existent cases.

People affected

• Occurrences reported: 1
• People reportedly harmed: 3
• People reportedly exposed: 6

Potential causes

Management

• Inadequate AI Training: Law firms failed to train staff on the proper and safe utilization of AI.
• Lack of Verification Policies: Management did not enforce mandatory verification steps for AI-sourced data.

Technology

• AI Hallucination: ChatGPT generated plausible but entirely fictitious legal cases and citations.
• Lack of Real-World Understanding: The LLM lacks actual comprehension of legal contexts and factual reality.

Data Inputs

• Training Data Limitations: The AI model was trained on datasets that limit its accuracy in niche contexts.

Human Factors

• Overzealous and Careless Use: Lawyers accepted AI outputs without verifying the accuracy of the citations.
• Uncritical Belief in AI: Attorneys relied blindly on ChatGPT to supplement critical legal research.
• Lack of Technical Familiarity: Lawyers used advanced AI tools without understanding their operational limits.

Process and Methods

• Lack of Independent Verification: Attorneys failed to cross-verify the AI-generated cases with official sources.
• Absence of Human-in-the-Loop: No robust review process was used to validate the AI outputs before sharing.

Regulatory Environment

• Lack of AI Legal Standards: Absence of clear regulatory standards on using generative AI in legal briefs.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts of the South African court incident, including quotes from the magistrate's judgment, details of how the fabrication was discovered, and the consequences (punitive costs and delay). The facts are well-corroborated across multiple news sources.
• Ambiguities identified: None significant; the sequence of events and the role of ChatGPT are clearly defined.
• Alternative interpretations: None.

A South African legal team presented fictitious, ChatGPT-generated case citations in a defamation lawsuit, leading to a two-month postponement and punitive costs. This incident highlights the professional risks of generative AI hallucinations and human overreliance, but carries negligible national security implications.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: South Africa
• Alleged perpetrator: Plaintiff's legal team

Threat characteristics

• Imminence: Long-term. Represents an ongoing, long-term challenge regarding the reliability of generative AI in professional settings rather than an active national security crisis.
• Autonomy: Human-controlled. The AI system acted as a research assistant, and human lawyers retained final control over whether to submit the generated text to the court.
• Novelty: Evolved capability. Represents a significant advancement of LLM hallucinations entering formal judicial proceedings, establishing a concerning precedent for professional reliance.

Impact by dimension

• Physical security: Negligible. No physical infrastructure, systems, or human safety threats were involved in this localized legal research error.
• Information security: Negligible. The incident involved unintentional legal research errors rather than systematic information warfare, intelligence compromise, or state-sponsored disinformation.
• Sovereignty: Negligible. While a local court proceeding was delayed by two months, this was a minor civil matter with no impact on core national sovereignty or critical government functions.
• Economic security: Negligible. Economic impact was limited to private legal fees and punitive court costs, with no threat to strategic industries or national economic security.
• Societal stability: Negligible. No threats to societal stability, civil liberties, or population safety occurred as a result of this isolated legal hallucination.