Sichuan Province Beset by Numerous Fabricated AI-Generated Reports of Disasters and Crises

In Sichuan, AI tools were reportedly used to create and disseminate false reports across various cities, including fabricated stories of landslides, earthquakes, standoffs, accidents, and health-related incidents. These AI-driven rumors misled the public, caused alarm and confusion, and strained local authorities. The alleged perpetrators sought social media engagement and financial gain through clickbait content. The authorities in Sichuan ultimately levied administrative penalties against the perpetrators.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The primary issue is the intentional generation and dissemination of false and misleading news articles and images using AI tools, which misled the public.

Additional risk subdomains

• 3.2 Pollution of information ecosystem and loss of consensus reality: One perpetrator generated over 420,000 fake articles at scale, significantly polluting the online information ecosystem.
• 4.3 Fraud, scams, and targeted manipulation: The perpetrators generated these false rumors to gain personal financial advantage through traffic monetization.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incidents were caused by human actors who intentionally used deployed AI tools to generate and spread false rumors for personal profit.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. The incident involves the use of AI to generate synthetic text and images (deepfakes/fake news), which falls under transparency obligations for AI-generated content.

AI system and alleged parties

• AI system: AI千文, unspecified, 智普清言, 翠绿笔墨 (Zhipu AI)
• AI purpose: Social Media Content Generation; Deepfake Image Generation
• Behaviour type: Tool
• Alleged developer: Unknown deepfake technology developers, Unknown AI developers
• Alleged deployer: Zhu Mou, Zhou Moumou, Zhang Moumou, Yang Moumou, Wang Mou, Shang Moumou, Luo Moumou, Liu Mou, Ji Moumou, Chen Mou
• Alleged harmed parties: Sichuan general public, Sichuan authorities

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Substantial, indirect Substantial
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the creation and spread of malicious online rumors and fabricated stories.

Directly caused: Perpetrators used AI tools to generate fake stories about deaths, disasters, and social conflicts to attract traffic.

Psychological

Reported: The report explicitly mentions that the fabricated rumors caused social panic and public anxiety.

Indirectly caused: The false reports of natural disasters, such as earthquakes and landslides, and violent standoffs caused public panic and anxiety among local residents.

Inferred additional harm: It is highly likely that thousands of local residents experienced temporary distress, fear, or anxiety upon reading realistic but entirely fabricated reports of disasters in their immediate areas.

Epistemic

Reported: The report explicitly describes extensive epistemic harm through the generation of fake news, fabricated disaster reports, and false images.

Directly caused: Perpetrators generated and distributed approximately 420,000 fake articles and multiple false news stories, misleading online users and eroding trust in digital information.

Indirectly caused: The widespread dissemination of these rumors disrupted public order and undermined the shared consensus reality regarding local events.

Inferred additional harm: The massive scale of AI-generated misinformation online makes it increasingly difficult for the public to distinguish between real and fabricated news, leading to long-term epistemic degradation.

People affected

• Occurrences reported: 10
• People reportedly exposed: 420000

Potential causes

Management

• Platform Reward Loops: Platform monetization algorithms incentivized sensationalist AI content.
• Lack of AI Tool Guardrails: Developers of AI tools failed to implement strict content safety filters.
• Volunteer Network Exploitation: Schemes managed large groups of volunteers to distribute AI-generated rumors.

Technology

• AI Text and Image Generators: AI software lowered the technical barrier to generate realistic fake content.
• Automated Content Generation Tools: Tools like 'Cuilv Bimo' enabled bulk generation of fake articles.
• Realism of AI-Synthesized Media: AI-edited videos and images easily deceived viewers into believing rumors.

Data Inputs

• Misappropriated Real-world Media: Real photos and foreign videos were used as inputs to seed AI generation.
• Unverified Online Information: Unverified news and rumors were fed into AI tools to synthesize fake stories.
• Unfiltered Prompt Inputs: AI tools processed false or misleading prompts without truth verification.

Human Factors

• Profit-Driven Motivation: Creators fabricated AI rumors to attract traffic, clicks, and rewards.
• Lack of Ethical Awareness: Users generated harmful AI content without considering public panic.
• Credulity of Online Audiences: Netizens easily believed and shared realistic AI-generated fake news.

Process and Methods

• Automated Batch Posting Systems: Systems automated draft creation and posting workflows for AI rumors.
• Synthesis and Editing Methods: Grafting real images onto unrelated AI stories misled the public.
• Inadequate Platform Filtering: Platform mechanisms failed to detect and block AI-generated rumors.

Regulatory Environment

• Lagging AI Content Regulations: Regulations on labelling AI-generated content were not fully preventative.
• Post-hoc Enforcement: Law enforcement acted after the AI rumors had already spread and caused panic.
• Lack of Platform Accountability: Platforms lacked strict penalties for hosting AI-generated disinformation.

Information quality

• Classification confidence: High
• Reason for confidence: The report from the Sichuan Provincial Public Security Department is highly detailed, outlining ten specific cases with names of perpetrators, locations, AI tools used, and the exact nature of the rumors generated.
• Ambiguities identified: The exact technical mechanisms of the 'Cuilv Bimo' tool are not fully detailed, but its function as an automated generator is clear.
• Alternative interpretations: None, as the police investigation clearly established the intentional use of AI to generate rumors.

Chinese domestic actors utilized generative AI tools to fabricate and disseminate over 420,000 fake articles, including false reports of natural disasters and violence, to generate ad revenue. While causing localized public anxiety and social panic, the incident represents a minor national security threat focused on profit-driven misinformation rather than coordinated state-sponsored information warfare.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: China
• Alleged perpetrator: Domestic individuals

Threat characteristics

• Imminence: Long-term. The specific cases have been resolved by law enforcement, but the scalable capability to generate fake news represents an ongoing strategic concern.
• Autonomy: Human-controlled. The AI tools functioned as content generators under the direct command and orchestration of human operators who initiated the prompts and set up the automated posting accounts.
• Novelty: Established threat. The use of large language models to generate spam, misleading articles, and fake news for clickbait and profit is a well-documented and established threat pattern.

Impact by dimension

• Physical security: Negligible. The incidents involved purely digital fabrication of news stories without physical attacks, direct kinetic impact, or compromise of critical infrastructure.
• Information security: Minor. While 420,000 fake articles were generated using tools like 'AI Qianwen' and 'Zhipu Qingyan', the operations were conducted by domestic actors for financial profit rather than a coordinated state-sponsored foreign intelligence campaign.
• Sovereignty: Negligible. The incidents did not target or compromise state authority, government decision-making, or electoral systems, and were quickly resolved by local police through administrative penalties.
• Economic security: Negligible. The perpetrators sought minor financial gain through ad-revenue and traffic monetization, with no strategic economic theft, financial system disruption, or technological security compromise.
• Societal stability: Minor. The fabricated reports of local disasters and violent standoffs caused temporary public anxiety and localized social panic, but did not lead to widespread civil unrest or systemic human rights violations.