Bots Allegedly Made up Roughly Half of Twitter Accounts in Discussions Surrounding COVID-19 Related Issues

Bots by anonymous actors were found by researchers to make up roughly half of Twitter accounts participating in COVID-19 discussions, many of which posted tweets about “reopening America“.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale

The incident involves the coordinated use of bot networks to conduct a large-scale influence campaign aimed at manipulating public opinion regarding pandemic policies.

Additional risk subdomains

• 3.2 Pollution of information ecosystem and loss of consensus reality: The massive influx of automated tweets polluted the social media ecosystem, distorting the perceived consensus on public health measures.
• 3.1 False or misleading information: The bots actively spread inaccurate COVID-19 stories and conspiracy theories to Twitter users.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The coordinated bot campaign was intentionally deployed by human actors (interest groups or countries) to manipulate public opinion post-deployment.

EU AI Act risk tier

• Risk tier: 1 Unacceptable

Risk Level 1: Unacceptable Risk. The report describes the use of AI-driven bot networks for 'subliminal manipulation or behavioral influence leading to harm' by artificially inflating political movements and spreading health misinformation.

AI system and alleged parties

• AI system: artificial-intelligence systems
• AI purpose: Social Media Content Generation; Content Recommendation
• Behaviour type: Multi-agent
• Alleged developer: unknown
• Alleged deployer: unknown
• Alleged harmed parties: Twitter users participating in COVID-19 discussions, Twitter Users, Twitter

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Severe. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Severe, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Minor, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Severe, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the spread of inaccurate COVID-19 stories and conspiracy theories by bot accounts.

Directly caused: Bots spread over 100 types of inaccurate COVID-19 stories and conspiracy theories across Twitter.

Inferred additional harm: Millions of Twitter users were likely exposed to toxic health misinformation and conspiracy theories.

Democracy

Reported: The report explicitly describes the artificial inflation of a political movement to influence public policy.

Directly caused: Bots artificially inflated the 'reopen America' movement, distorting public debate and political agendas during a health crisis.

Inferred additional harm: The manipulation of public discourse likely undermined democratic decision-making processes regarding pandemic responses.

Epistemic

Reported: The report explicitly describes epistemic harm through the spread of false information and the pollution of the information ecosystem.

Directly caused: Bots generated and amplified false narratives about COVID-19, making it difficult for users to distinguish real public opinion from automated propaganda.

Inferred additional harm: The widespread dissemination of false health information likely eroded public trust in scientific consensus and official health guidelines.

People affected

• Occurrences reported: 1
• People reportedly exposed: 200000000

Potential causes

Management

• Slow Moderation Response: Platform management failed to quickly address the surge in COVID-19 bots.
• Prioritizing Features Over Safety: Testing features that hinder moderation instead of prioritizing safety.
• Lack of Platform Transparency: Failure to openly address and mitigate coordinated bot campaigns.

Technology

• High Availability of Botnets: Explosion of botnets for hire makes launching mass campaigns easy.
• Sophisticated Bot Automation: Bots mimic human behavior by copying text and timing posts.
• Flawed Platform Reply Controls: New reply controls tested by Twitter make slowing misinformation harder.

Data Inputs

• Copied and Pasted Messaging: Identical text and hashtags are spread rapidly across bot networks.
• Spoofed Location Data: Bots fake locations to appear as local community members.
• High-Frequency Tweet Streams: Accounts post faster than humanly possible to dominate debates.

Human Factors

• Human Amplification via Bots: Real people use bot assistants to artificially spread their messages.
• More Time to Build Botnets: Stay-at-home orders gave actors more free time to build networks.
• Lack of User Vigilance: Users trust accounts without verifying if they are real people.

Process and Methods

• Inadequate Bot Detection: Twitter lacks robust automated systems to weed out active bots.
• Inability to Track Bot Origins: No clear process to trace if bot activity is foreign or domestic.
• Weak Account Verification: Platform processes fail to verify the true identity of accounts.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, consistent findings from a reputable academic study by Carnegie Mellon University, detailing the scale of the bot activity and its impact on public discourse.
• Ambiguities identified: The exact geographic origins of the botnets and the specific actors behind them remain unknown.
• Alternative interpretations: Some might view the bots as simple tools rather than a coordinated multi-agent system, but the scale and synchronized nature of the campaign support the multi-agent classification.

An analysis of Twitter data during the early COVID-19 pandemic revealed that automated bot accounts drove approximately half of the conversation regarding reopening the country. This large-scale, coordinated campaign distorted public discourse and amplified health misinformation, illustrating the evolving threat of AI-driven information warfare to societal stability and public health policy.

• Overall national security impact: Substantial
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The incident represents an ongoing, long-term strategic threat of automated influence operations rather than an active, immediate tactical emergency.
• Autonomy: Human-supervised. While many bots operated automatically, they were largely directed by human actors and used as assistants to amplify specific human-generated political narratives.
• Novelty: Evolved capability. Social media manipulation via bots is an established threat, but the scale and intensity of targeting a global pandemic represents a significant evolution in capability.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or kinetic operations were targeted or affected by the social media bot activity.
• Information security: Substantial. The massive scale of the bot campaign, accounting for half of the discussion, represents a substantial information warfare effort to manipulate public opinion during a national crisis.
• Sovereignty: Minor. The campaign aimed to artificially influence domestic political movements and policy decisions regarding pandemic reopening, but did not directly compromise core government systems.
• Economic security: Negligible. No significant economic manipulation or theft of critical technological assets was reported in connection with this incident.
• Societal stability: Substantial. The widespread dissemination of health misinformation and conspiracy theories threatened societal stability and public trust during a national health emergency.