China Reportedly Intensifying AI to Spread Disinformation to U.S. and Taiwanese Voters

AI tools linked to China were used to disseminate disinformation targeting voters in the U.S. and Taiwan, according to a Microsoft report. These operations included AI-generated imagery and audio to influence political perceptions and election outcomes, originating from the APT Storm-1376 (also known as Spamouflage and Dragonbridge).

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale

The incident involves state-sponsored actors using generative AI to conduct coordinated, large-scale disinformation and influence campaigns targeting democratic elections in the U.S. and Taiwan.

Additional risk subdomains

• 3.2 Pollution of information ecosystem and loss of consensus reality: The massive scaling of low-quality AI-generated articles, comments, and cartoons across social media platforms threatens to pollute the information ecosystem.
• 3.1 False or misleading information: The campaigns generated specific false narratives, such as fabricated voter fraud claims and fake audio endorsements.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The influence operations were intentionally designed and executed by human threat actors (state-sponsored groups from Russia, China, and Iran) using post-deployment commercial AI systems to scale their campaigns.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the use of chatbots (ChatGPT) and the generation of deepfakes (voice clones, fake audio clips), which fall under the transparency obligations of Risk Level 3.

AI system and alleged parties

• AI system: AI-generated news anchors, ChatGPT, DALL-E 3, generative AI tools (ByteDance, OpenAI)
• AI purpose: Social Media Content Generation; Voice Generation
• Behaviour type: Assistant
• Alleged developer: Storm-1376, Spamouflage, Dragonbridge, Chinese Communist Party
• Alleged deployer: Storm-1376, Spamouflage, Dragonbridge, Chinese Communist Party
• Alleged harmed parties: U.S. voters, Taiwanese voters, General public, Election integrity, Democracy

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Minor
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Minor, indirect Minor
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Substantial
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the creation and spread of toxic or malicious content directly caused by the incident.

Directly caused: AI tools like ChatGPT, DALL-E 3, and ByteDance tools were used to generate fake news articles, political cartoons, and deepfake audio clips.

Indirectly caused: These AI-generated assets were distributed across social media platforms (X, Facebook, Telegram) to target voters.

Inferred additional harm: It is highly likely that additional unmonitored fake accounts and websites generated similar toxic or polarizing content that went undetected.

Democracy

Reported: The report explicitly describes attempts to undermine democratic norms directly caused by the incident.

Directly caused: AI was used to scale influence operations, generate fake endorsements (Taiwan), and clone the FBI director's voice to allege voter fraud, directly targeting democratic processes.

Indirectly caused: The proliferation of these campaigns forced governments and platforms to expend significant resources to monitor, expose, and debunk the operations.

Inferred additional harm: Continued use of AI in this manner could lead to a long-term erosion of public trust in democratic institutions and election integrity.

Epistemic

Reported: The report explicitly describes epistemic harm directly caused by the incident.

Directly caused: Generative AI was used to rewrite plagiarized articles, fabricate claims, and create realistic fake audio clips of political figures.

Indirectly caused: These fabrications misled fact-checkers over 250 times and polluted social media feeds with low-quality, misleading content.

Inferred additional harm: The widespread availability of these tools likely contributed to a broader degradation of the shared information ecosystem, making it harder for average citizens to distinguish real from synthetic media.

People affected

• Occurrences reported: 1
• People reportedly exposed: 500000

Potential causes

Management

• Inadequate Platform Moderation: Tech platforms failed to proactively block automated text generation comments.

Technology

• AI Voice Cloning: Operation Overload used AI to clone FBI Director's voice to claim voter fraud.
• Generative AI Content Rewriting: CopyCop used ChatGPT and DALL-E 3 to rewrite articles and create fake sites.
• AI-Generated News Anchors: Spamouflage used ByteDance AI tools to create fake news anchors in videos.

Data Inputs

• Plagiarized Content Input: Generative AI models ingested real news to rewrite and spin disinformation.
• Lack of Shared Technical Indicators: Absence of shared AI indicators hinders rapid detection across platforms.

Human Factors

• Susceptibility to Fake Audio: Audiences are easily duped by highly convincing AI-generated audio clips.
• Voter Polarization Exploitation: Adversaries used AI content to exploit and deepen existing domestic divisions.

Process and Methods

• Inefficient Fake Account Prevention: Social platforms fail to prevent rapid creation of automated bot accounts.
• Lack of Rapid AI Verification: Inability to quickly verify and identify AI-generated content at scale.

Regulatory Environment

• Weak Hosting KYC Processes: Adversaries easily access Western hosting services due to loose identity checks.

Information quality

• Classification confidence: High
• Reason for confidence: The reports from FDD and Microsoft provide detailed, well-researched accounts of multiple state-sponsored operations, identifying specific AI tools (ChatGPT, DALL-E 3, ByteDance tools) and their exact roles in generating content. The findings are corroborated by multiple government agencies (CISA, FBI, ODNI) and independent research groups (Clemson University, Recorded Future, ISD).
• Ambiguities identified: The exact scale of the audience reached by some of the AI-generated content remains difficult to measure precisely, as noted by the researchers.
• Alternative interpretations: None. The role of AI as a scaling tool for human-driven disinformation is clearly established.

State-sponsored actors from Russia, China, and Iran integrated generative AI tools to scale covert influence operations targeting the 2024 US and Taiwanese elections. While the operations failed to achieve measurable impact due to proactive defense and the resilience of the information ecosystem, they represent a significant, ongoing information warfare and sovereignty threat.

• Overall national security impact: Substantial
• Scope: Multiple nations
• Primary target: United States
• Other affected: Taiwan
• Alleged perpetrator: Russia, China, and Iran

Threat characteristics

• Imminence: Long-term. The immediate election threat has passed, but the use of generative AI by foreign adversaries remains an ongoing strategic concern.
• Autonomy: Human-controlled. AI tools were used as assistants to generate content, while human operators directed the campaigns and deployed the assets.
• Novelty: Evolved capability. The incident represents an evolved capability where existing foreign influence techniques were scaled using generative AI tools.

Impact by dimension

• Physical security: Negligible. No physical security threats, kinetic attacks, or critical infrastructure compromises were reported in connection with these operations.
• Information security: Substantial. Foreign state-sponsored actors from Russia, China, and Iran launched coordinated information warfare operations using generative AI to scale disinformation, though with limited effectiveness.
• Sovereignty: Substantial. The operations directly targeted the democratic electoral systems of the US and Taiwan, attempting to interfere with core sovereign government processes.
• Economic security: Negligible. No strategic technology theft or financial system attacks occurred, though commercial AI systems were exploited by foreign adversaries.
• Societal stability: Minor. Attempts were made to deepen political polarization and manipulate public trust, but the operations failed to achieve measurable societal impact.