Microsoft Copilot Falsely Accuses Journalist Martin Bernklau of Crimes

Microsoft's Copilot is reported to have falsely accused veteran court reporter Martin Bernklau of committing serious crimes, including child abuse and fraud. The tool is described as having generated defamatory content that not only accused Bernklau of multiple crimes he covered as a journalist but also provided his personal contact details. Attempts by Microsoft to remove the false entries were only temporarily successful, as the defamatory information reportedly reappeared.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The AI system generated false and defamatory accusations against a journalist, confusing him with the criminals he had reported on, leading to severe emotional distress and reputational risk.

Additional risk subdomains

• 2.1 Compromise of privacy by obtaining, leaking or correctly inferring sensitive information: The AI system exposed the journalist's private contact details, including his full address and phone number, without consent.
• 7.3 Lack of capability or robustness: The system failed to perform reliably, hallucinating false narratives and failing to maintain permanent deletions of the incorrect data.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The risk was caused by the AI system's generation of false information post-deployment, which was an unintended outcome of its design.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system is a chatbot (Microsoft Copilot), which falls under Risk Level 3 due to transparency obligations requiring users to be informed they are interacting with an AI.

AI system and alleged parties

• AI system: Copilot (Microsoft)
• AI purpose: Chatbot; Content Search
• Behaviour type: Assistant
• Alleged developer: Microsoft
• Alleged deployer: Microsoft Copilot, Microsoft
• Alleged harmed parties: Martin Bernklau

Harm severity

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes, the report describes toxic and defamatory content generated by the AI.

Directly caused: The AI generated text falsely accusing the journalist of child abuse, exploiting dependents, escaping a psychiatric hospital, and defrauding widows.

Indirectly caused: N/A

Inferred additional harm: N/A

Civil rights

Reported: Yes, the report describes violations of privacy and data protection rights under GDPR.

Directly caused: The AI violated the journalist's right to have accurate personal data processed and his right to be forgotten/rectified under the GDPR.

Indirectly caused: N/A

Inferred additional harm: Many other EU citizens likely have their GDPR rights violated by LLMs that process inaccurate personal data without a viable mechanism for correction or deletion.

Privacy

Reported: Yes, the report explicitly describes privacy violations.

Directly caused: The AI system exposed the journalist's full private address, phone number, and a route planner to his home.

Indirectly caused: N/A

Inferred additional harm: Other individuals searched on the platform may have had their private contact details exposed without consent.

Psychological

Reported: Yes, the report explicitly describes psychological harm to the journalist.

Directly caused: Bernklau described the experience of seeing his name associated with serious crimes as traumatizing, causing 'a mixture of shock, horror, and disbelieving laughter.'

Indirectly caused: N/A

Inferred additional harm: The judge and other individuals falsely accused likely experienced similar distress and anxiety regarding their reputation and safety.

Epistemic

Reported: Yes, the report explicitly describes epistemic harm through AI-driven misinformation and fabrication.

Directly caused: Microsoft Copilot fabricated false criminal histories for the journalist and generated a list of hallucinated, non-existent article titles.

Indirectly caused: N/A

Inferred additional harm: The widespread use of LLMs as search tools risks polluting the information ecosystem with convincing but entirely fabricated facts and citations.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 2

Potential causes

Management

• Inadequate Incident Response: Microsoft applied temporary blocks that failed after a few days.
• Reliance on Disclaimers: Terms of service disclaim liability instead of guaranteeing accuracy.

Technology

• Entity Association Failure: Model confused the court reporter with the criminals in his reports.
• Probabilistic Output: LLM generates text based on statistical probability, not factual truth.
• Source Hallucination: System fabricated sources and article titles to support false claims.

Data Inputs

• Unstructured Public Data: Decades of online court reports linked the journalist to crime terms.
• Irrelevant Source Linking: Search results returned unrelated videos and articles for the name.

Human Factors

• User Overreliance: Users assume search chatbots provide verified and factual information.

Process and Methods

• Ineffective Data Deletion: Microsoft failed to permanently delete the false information from LLM.
• Lack of Output Verification: No real-time fact-checking process exists before displaying results.

Regulatory Environment

• Legal Accountability Gap: Prosecutors rejected charges because the AI is not a legal person.
• GDPR Enforcement Deficit: Current regulations struggle to enforce data rectification in LLMs.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts of the specific false accusations, the journalist's reaction, the technical reasons for the failure, and the legal/regulatory context. There are no major contradictions.

Microsoft Copilot generated defamatory statements and leaked private contact details of a German journalist due to LLM hallucinations. While causing personal and psychological distress, the incident presents negligible direct threat to national security, highlighting instead systemic challenges in generative AI data accuracy and GDPR compliance.

• Overall national security impact: Minor
• Response level: Moderate
• Scope: Single nation
• Primary target: Germany
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding AI data accuracy and regulatory compliance rather than an immediate crisis.
• Autonomy: Human-supervised. The AI generates responses autonomously, but operates under the developer's oversight and system-level intervention capabilities, though enforcement of data deletion was flawed.
• Novelty: Established threat. LLM hallucinations, data leaks, and difficulties with right-to-erase requests are well-documented and established issues in generative AI.

Impact by dimension

• Physical security: Negligible. No threat to physical systems, infrastructure, or human safety was identified in this incident.
• Information security: Negligible. The incident involves localized AI hallucinations rather than a coordinated disinformation campaign or compromise of intelligence capabilities.
• Sovereignty: Negligible. No threat to state authority, electoral systems, or core government operations was observed.
• Economic security: Negligible. The incident does not pose a threat to strategic industries, financial systems, or national economic security.
• Societal stability: Minor. The AI system violated individual privacy rights by exposing contact details and generating defamatory content, highlighting systemic challenges in enforcing GDPR compliance for LLMs.