NewsBreak Reportedly Published AI-Assisted Articles Containing Purportedly False Local and Community-Service Information

NewsBreak reportedly published AI-assisted local articles containing purportedly false information, including a fabricated Christmas shooting in Bridgeton, New Jersey, incorrect Food to Power distribution times that caused people to be turned away, and nonexistent Harvest912 foot-care clinics. NewsBreak said the errors originated in source material and removed the articles after complaints. The specific AI models and the extent to which they contributed to the inaccuracies were not identified.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The AI system generated and spread false local news stories, including a fabricated shooting and incorrect charity event details, which misled readers and caused real-world disruption.

Additional risk subdomains

• 6.3 Economic and cultural devaluation of human effort: NewsBreak used AI to scrape and rewrite competitors' stories under fictitious bylines, leading to copyright lawsuits and displacing human journalistic effort.
• 7.3 Lack of capability or robustness: The AI system failed to reliably and accurately process input data, leading to hallucinations and factual errors in critical local information.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The false stories and incorrect information were generated by deployed AI tools rewriting scraped content, which was an unexpected and unintended outcome of the system's operation.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system generates synthetic news content. Under the Act, such systems pose a moderate risk and require transparency obligations, such as notifying users that the content is AI-generated.

AI system and alleged parties

• AI system: OpenAI
• AI purpose: Automated Content Curation; Writing Assistant
• Behaviour type: Tool
• Alleged developer: NewsBreak
• Alleged deployer: NewsBreak, News organizations
• Alleged harmed parties: Police departments, People seeking food assistance, People experiencing homelessness, News consumers, Journalistic integrity, Harvest912, Food to Power, Epistemic integrity, Community service organizations, Bridgeton Police Department

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Substantial, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: The report explicitly describes financial losses from copyright infringement lawsuits, including a $1.75 million settlement with Patch Media.

Directly caused: NewsBreak paid a $1.75 million settlement to Patch Media for copyright infringement. Across the 40 reported instances of AI-related issues, this averages to $43,750 per occurrence.

Inferred additional harm: NewsBreak likely incurred additional undisclosed financial losses from other settlements, such as the one with Emmerich Newspapers, and associated legal fees.

Epistemic

Reported: The report explicitly describes epistemic harm through the generation and dissemination of fabricated news stories and incorrect local information.

Directly caused: The AI system fabricated a shooting in Bridgeton, NJ, and published incorrect schedules for food banks and homeless clinics, corrupting the local information environment.

Indirectly caused: The dissemination of false stories eroded community trust in local news aggregators and forced local organizations to spend resources correcting the misinformation.

Inferred additional harm: Given NewsBreak's 50 million monthly users, it is highly likely that thousands of readers were misled by the 40+ erroneous AI-generated stories published since 2021.

People affected

• Occurrences reported: 40
• People reportedly harmed: 100
• People reportedly exposed: 50000

Potential causes

Management

• Prioritizing Growth Over Safety: Focus on filling local news voids rapidly led to systemic errors.
• Inadequate Risk Assessment: Management ran risky AI experiments without assessing brand damage.
• Lack of Editorial Oversight: Leadership failed to implement strict journalistic guidelines initially.

Technology

• AI Rewriting Tool Errors: AI algorithms generated fictional details when rewriting scraped local news.
• Flawed Scraping Algorithms: Algorithms scraped outdated or incorrect information from web pages.
• Use of Generative AI Experiments: OpenAI tools were used to generate articles under fake bylines.

Data Inputs

• Unverified Content Sources: Inaccurate source data from unverified websites like findplace.xyz.
• Outdated Charity Web Pages: Scraping tool used outdated food bank and clinic schedules from web pages.
• Scraped Competitor Content: Lifting snippets of competitor articles without proper verification.

Human Factors

• Lack of Journalistic Experience: Staff were new to journalism and did not verify AI-generated output.
• Unfamiliarity with US Market: Staff or management lacked understanding of US editorial standards.

Process and Methods

• Lack of Pre-Publication Review: No editorial review process to verify AI-generated stories before publishing.
• Ineffective Feedback Mechanisms: Customer support ignored early emails about erroneous charity schedules.
• Aggressive Scraping and Rewriting: Systematic rewriting of competitor content to fill local news gaps.

Regulatory Environment

• Weak Copyright Enforcement: Copyright laws did not deter unauthorized content rewriting initially.
• Lack of AI Content Regulations: No regulatory standards governing AI-generated local news accuracy.

Information quality

• Classification confidence: High
• Reason for confidence: The report from Reuters is highly detailed, providing specific dates, locations, names of affected charities, court settlement figures, and internal company memos. This allows for a highly confident classification.
• Ambiguities identified: The exact AI models used are not fully specified beyond a general mention of OpenAI, and the exact number of people misled by the false stories is not quantified.
• Alternative interpretations: The incident could also be viewed primarily as a copyright and journalistic ethics issue rather than an AI safety issue, though the central role of AI generation in creating the false content makes the AI safety classification highly appropriate.

NewsBreak, a US local news app with China-linked engineering, utilized AI tools to scrape and rewrite news, leading to over 40 fabricated or erroneous stories. While causing local confusion and financial settlements, the national security impact remains minor, primarily representing established risks of AI hallucination and lack of editorial oversight.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: NewsBreak

Threat characteristics

• Imminence: Long-term. Represents an ongoing, systemic issue with AI-generated content quality and moderation rather than an active, immediate national security crisis.
• Autonomy: Human-supervised. AI tools automatically scraped and rewrote articles, publishing them with minimal to no human editorial oversight, leading to hallucinations.
• Novelty: Established threat. AI hallucinations, scraping of copyrighted material, and dissemination of false information are well-documented and established issues in AI deployment.

Impact by dimension

• Physical security: Negligible. Fabricated news of a shooting was debunked by police, and incorrect charity event times did not cause physical damage or critical infrastructure disruption.
• Information security: Minor. AI-generated misinformation polluted local news feeds of a popular app with 50 million users, but lacked evidence of coordinated state-sponsored information operations.
• Sovereignty: Negligible. Local police had to debunk a fake shooting, but there was no significant disruption to core government functions, elections, or state sovereignty.
• Economic security: Minor. Caused copyright lawsuits and a 1.75 million dollar settlement for copyright infringement, but poses no broader threat to national economic or technological security.
• Societal stability: Minor. Misled vulnerable populations regarding charity services and created local confusion, but did not cause large-scale civil unrest or systemic rights violations.