Chinese Accounts Spammed Twitter Feed Allegedly to Obscure News of Protests

Twitter Feed was flooded by content from Chinese-language accounts which allegedly aimed to manipulate and reduce social media coverage about widespread protests against coronavirus restrictions in China.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale

The incident involved a coordinated campaign using automated accounts to conduct large-scale censorship and propaganda, aiming to manipulate public opinion and suppress news of political protests.

Additional risk subdomains

• 1.2 Exposure to toxic content: The spam campaign flooded search results with links to escort services and adult offerings, exposing users to inappropriate and pornographic content.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was driven by human actors who intentionally deployed automated spam accounts to suppress protest news, exploiting Twitter's post-deployment platform vulnerabilities.

EU AI Act risk tier

• Risk tier: 4 Minimal or No Risk

Risk Level 4: Minimal or No Risk. The incident involves automated spam accounts and Twitter's spam filters, which are classified as posing low or negligible risks to users and society under the Act.

AI system and alleged parties

• AI system: Twitter's anti-propaganda automations
• AI purpose: Social Media Content Generation; Spam Filtering
• Behaviour type: unknown
• Alleged developer: Twitter
• Alleged deployer: Twitter
• Alleged harmed parties: Twitter Users, Twitter

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Minor, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes a flood of adult content and escort service spam created and spread during the incident.

Directly caused: Numerous Chinese-language accounts spammed Twitter with links to escort services and other adult offerings alongside city names.

Inferred additional harm: Thousands of global Twitter users searching for news about Chinese cities were exposed to pornographic spam.

Civil rights

Reported: The report explicitly describes an attempt to suppress information about political protests, impacting freedom of expression.

Directly caused: The spam campaign successfully blocked information about the daring protests and calls for Communist Party leaders to resign for several hours.

Inferred additional harm: Protesters in China were temporarily prevented from sharing information globally, potentially impacting their safety and civil coordination.

Democracy

Reported: The report explicitly describes a coordinated campaign aimed at suppressing news of political protests.

Directly caused: The spam campaign obscured news of widespread protests against coronavirus restrictions and calls for political leaders to resign.

Inferred additional harm: The manipulation of public information on a major global platform undermines democratic expression and public assembly.

Epistemic

Reported: The report explicitly describes the pollution of the information ecosystem where searches for protest news returned useless spam.

Directly caused: For hours, anyone searching for posts from Chinese cities saw pages of useless tweets instead of information about the protests.

Inferred additional harm: The temporary loss of a shared consensus reality regarding the scale and occurrence of the protests in China.

People affected

• Occurrences reported: 1

Potential causes

Management

• Severe Staff Downsizing: Drastic headcount reduction weakened the platform's anti-propaganda defenses.
• Underestimation of Geopolitical Risk: Management did not adequately staff critical regional safety teams.

Technology

• Inadequate Automated Spam Filtering: Automated systems failed to flag and block coordinated Chinese spam campaigns.
• Vulnerable Search Algorithms: Search algorithms prioritized high-volume spam over legitimate protest news.

Data Inputs

• Dormant Accounts Reactivation: Lack of flags on suddenly active accounts spreading spam links.
• Spam Keyword Manipulation: Spammers manipulated location tags and keywords to drown out protest news.

Human Factors

• Mass Resignations of Analysts: All China-specific influence analysts resigned, leaving a human review gap.
• Staff Burnout and Reductions: Remaining staff were overwhelmed by the sudden high volume of spam.

Process and Methods

• Reliance on Manual Moderation: The mitigation process relied too heavily on manual intervention.
• Delayed Incident Response: Lack of proactive monitoring delayed the mitigation of the spam attack.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly describes the nature of the coordinated spam campaign, its timing, and its impact on Twitter searches during the Chinese protests. The role of automated accounts and the failure of Twitter's moderation systems are well-documented.
• Ambiguities identified: The exact technical nature of the automated accounts (whether they used advanced AI or simple scripting) is not specified.
• Alternative interpretations: The incident could be viewed purely as a human-driven political censorship campaign rather than an AI safety failure, though it directly exploited automated systems and failures in automated content moderation.

A coordinated campaign used automated accounts to flood Twitter search results with adult content, successfully obscuring news of widespread anti-lockdown protests in China. This incident highlights how adversaries can exploit vulnerabilities in private social media platform moderation to conduct effective information suppression and restrict global visibility of civil rights movements.

• Overall national security impact: Substantial
• Scope: Multiple nations
• Primary target: China
• Other affected: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The incident represents an ongoing, strategic threat of automated information manipulation rather than an immediate, active crisis requiring urgent national response.
• Autonomy: Human-supervised. The campaign utilized automated spam accounts to distribute content at scale, but was coordinated and directed by human actors.
• Novelty: Established threat. Using bot networks and spam to drown out political hashtags and search results is an established technique previously observed in information operations.

Impact by dimension

• Physical security: Negligible. The incident did not involve any threats to physical systems, critical infrastructure, or kinetic military action.
• Information security: Substantial. Coordinated automated accounts successfully conducted an information suppression campaign on a global platform, obscuring real-time crisis information during major political protests.
• Sovereignty: Minor. The campaign manipulated global information flow but did not directly compromise sovereign government functions or electoral systems of foreign nations.
• Economic security: Negligible. No significant threats to financial systems, strategic technologies, or critical supply chains were reported.
• Societal stability: Substantial. The operation directly restricted human and civil rights by suppressing political expression and coordination among protesters, temporarily blinding global observers to domestic civil unrest.