BNN Breaking's AI-Driven Errors Reportedly Damage Reputations and Spread Misinformation

BNN Breaking, an AI-driven news site, published a false story about Irish DJ Dave Fanning, damaging his reputation. The site used AI to generate error-filled content, leading to numerous complaints and a defamation lawsuit against BNN and Microsoft. The site has since gone dormant.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The AI system generated a false and defamatory news article that misidentified a prominent broadcaster, leading to significant reputational harm and a lawsuit.

Additional risk subdomains

• 3.2 Pollution of information ecosystem and loss of consensus reality: BNN used AI to churn out thousands of unverified, low-quality articles daily, polluting search engines and major platforms like MSN.
• 6.3 Economic and cultural devaluation of human effort: The AI tool was used to scrape and paraphrase human journalists' work, siphoning ad revenue away from legitimate local news outlets.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The immediate cause of the defamation was an AI association error during automated content generation, which was an unintentional outcome of BNN's deployment of the tool.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. The system is a generative AI tool used to produce news articles, which falls under the category of AI-generated content requiring transparency obligations so that users are aware the content is AI-generated.

AI system and alleged parties

• AI system: ChatGPT, ePiphany AI (OpenAI)
• AI purpose: Writing Assistant; Automated Content Curation
• Behaviour type: Assistant
• Alleged developer: Gurbaksh Chahal, ePiphany AI
• Alleged deployer: BNN Breaking
• Alleged harmed parties: Journalism, General public, Dave Fanning, BNN Breaking readers

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Minor
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Minor
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Minor, indirect Minor
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: The report describes indirect financial losses to local news outlets and legal costs for the defamed broadcaster, but does not quantify a specific total dollar amount.

Indirectly caused: Local news outlets like the Limpopo Mirror lost advertising revenue due to BNN's AI-generated content siphoning traffic, and [PERSON_002] incurred legal costs filing a defamation lawsuit.

Inferred additional harm: Substantial cumulative financial losses for legitimate news publishers due to diverted programmatic ad spend, and significant legal expenses for individuals defamed by BNN's automated output.

Malicious content

Reported: The report explicitly describes the creation and spread of defamatory content falsely accusing an individual of sexual misconduct.

Directly caused: The AI system generated a defamatory article pairing [PERSON_002]'s photo with allegations of sexual misconduct, which was promoted on MSN's homepage.

Inferred additional harm: Other harmful or defamatory articles may have been generated among the thousands of unvalidated daily posts.

Democracy

Reported: The report describes the erosion of public trust in legitimate journalism and the contamination of the information supply.

Indirectly caused: The mass-production of AI-generated falsehoods promoted by major platforms like MSN and Google News erodes public trust in journalism and democratic information ecosystems.

Inferred additional harm: Continued proliferation of automated spam news sites could severely undermine the shared consensus reality necessary for democratic processes.

Privacy

Reported: The report explicitly describes the unauthorized use of journalists' real identities on AI-generated articles.

Directly caused: The AI system randomly assigned real journalists' names as bylines to unverified, AI-generated articles without their consent.

Psychological

Reported: The report explicitly describes reputational damage and distress to the defamed broadcaster and the journalists.

Directly caused: [PERSON_002] experienced significant distress and reputational damage due to being falsely associated with sexual misconduct.

Indirectly caused: BNN journalists experienced distress and career jeopardy due to unauthorized bylines on AI-generated errors.

Inferred additional harm: Other individuals falsely accused or misquoted by BNN's thousands of automated articles likely suffered similar reputational distress.

Epistemic

Reported: The report explicitly describes extensive epistemic harm, including fabricated quotes, invented panelists, and false factual claims.

Directly caused: The AI system fabricated quotes from a lawyer, invented a panelist for a literature festival story, and made false geographical claims about Dungeness crabs.

Indirectly caused: These AI-generated falsehoods were indexed by Google News and republished by MSN, misleading readers and polluting the broader information ecosystem.

Inferred additional harm: Widespread public confusion and erosion of trust in digital news due to the automated dissemination of thousands of unverified articles.

People affected

• Occurrences reported: 1
• People reportedly harmed: 51
• People reportedly exposed: 10000000

Potential causes

Management

• Hostile Management Practices: Founder dismissed employee concerns about AI errors and integrity.
• Growth-First Business Model: Management focused on scaling output via AI to maximize ad revenue.

Technology

• AI Hallucination in Image Pairing: AI paired text with generic photo of a prominent Irish broadcaster.
• ePiphany AI Chatbot Errors: The AI tool generated paraphrased stories containing factual errors.
• Automated Content Curation: Microsoft replaced human editors with AI systems to curate MSN homepage.

Data Inputs

• Unverified Source Data: AI uploaded articles from other sites to paraphrase without verification.
• Lack of Proper Image Metadata: AI matched text to generic images lacking precise identifier metadata.

Human Factors

• Lack of Manual Validation: Staff failed to check stories due to the high volume of AI output.
• Forced Byline Attribution: Founder forced journalists' bylines on unchecked AI-generated stories.

Process and Methods

• Prioritizing Speed Over Accuracy: Process optimized for high-volume automated publishing over factual checks.
• Inadequate Content Review: Microsoft lacked rigorous human-in-the-loop review for licensed feed.
• Algorithmic Monetization: Programmatic advertising incentivized mass-producing AI clickbait.

Regulatory Environment

• Lack of AI Disclosure Rules: No strict regulations requiring clear disclosure of AI-generated news.
• Ineffective Platform Oversight: Search engines and portals lacked strict standards for AI news sites.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides comprehensive, first-hand accounts from former BNN employees, the defamed individual, and screenshots of internal communications, leaving little ambiguity about the events.
• Ambiguities identified: The exact technical architecture of ePiphany AI (whether it was a wrapper of ChatGPT or a custom model) is not fully verified, though employees assumed it licensed existing software.

A commercial news site, BNN Breaking, used generative AI to mass-produce fabricated news stories for ad revenue, resulting in defamation and information pollution. While representing a significant evolution in AI-enabled misinformation, the incident has minor national security implications as it lacks state-sponsored coordination or targeting of critical national infrastructure.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Ireland, United States, Hong Kong
• Alleged perpetrator: Gurbaksh Chahal and BNN Breaking

Threat characteristics

• Imminence: Long-term. The specific site is dormant, but the proliferation of AI-generated content farms remains an ongoing strategic concern.
• Autonomy: Human-supervised. AI generated content with minimal human oversight, bypassing intended manual validation steps due to high volume.
• Novelty: Evolved capability. Represents an evolution of established spam-farm techniques using modern generative AI to mass-produce fabricated news.

Impact by dimension

• Physical security: Negligible. No impact on physical systems, critical infrastructure, or human safety.
• Information security: Minor. AI-generated false news and fabricated quotes polluted the information ecosystem on MSN, but lacked state-sponsored coordination or targeted intelligence compromise.
• Sovereignty: Negligible. No disruption to core government functions, sovereignty, or electoral systems.
• Economic security: Negligible. Minor financial impact on local news publishers and legal costs for defamed individuals, with no threat to strategic industries or financial systems.
• Societal stability: Minor. Reputational harm to specific journalists and a broadcaster, but no large-scale threat to social cohesion or civil liberties.