Manipulated Media via AI Disinformation and Deepfakes in 2024 Elections Erode Trust Across More Than 50 Countries

AI-driven election disinformation is escalating globally, leveraging easy-to-use generative AI tools to create convincing deepfakes that mislead voters. This shift has simplified the process for individuals to generate fake content, having already eroded trust in elections by undermining public trust and manipulating voter perceptions. Evidence has, for example, been documented in incidents across the U.S., Moldova, Slovakia, Bangladesh, and Taiwan.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale

The incident involves coordinated, large-scale disinformation campaigns using AI-generated deepfakes to influence political processes and public opinion across multiple countries.

Additional risk subdomains

• 3.2 Pollution of information ecosystem and loss of consensus reality: The deluge of deepfakes erodes public trust in what they see and hear, threatening shared reality.
• 1.2 Exposure to toxic content: In Bangladesh, an opposition lawmaker was depicted in a bikini, exposing conservative audiences to non-consensual sexualized content.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The risk is caused by human actors intentionally using post-deployment generative AI tools to create and spread political disinformation.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the creation and spread of AI-generated content such as deepfakes, which requires transparency obligations under the EU AI Act.

AI system and alleged parties

• AI system: unspecified (Google, OpenAI)
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: Unknown deepfake creators, OpenAI, Google
• Alleged deployer: Russian government, Political operatives, Political consultants, Chinese Communist Party
• Alleged harmed parties: Voters, Public trust, Political figures, General public, Electoral integrity, Democracy, Civic society

Harm severity

Highest direct severity in any category: Severe. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Severe
• Differential treatment: direct Minor, indirect Minor
• Civil rights: direct Minor, indirect Substantial
• Democracy: direct Minor, indirect Substantial
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Minor, indirect Minor
• Epistemic: direct Minor, indirect Severe
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes, the report describes the creation of deepfake videos and audio clips designed to mislead voters.

Directly caused: Deepfakes of Moldova's president, Slovakia's liberal leader, and Bangladesh's opposition lawmaker wearing a bikini were created and spread.

Indirectly caused: These deepfakes were widely shared on social media platforms like Facebook and TikTok, reaching millions of users.

Inferred additional harm: Numerous other unmentioned deepfakes were likely generated and circulated during these election cycles.

Differential treatment

Reported: Yes, female politicians like Rumeen Farhana were targeted with sexualized deepfakes, reflecting gender-based harassment.

Directly caused: Rumeen Farhana was falsely depicted in a bikini to spark outrage in a conservative, majority-Muslim nation.

Indirectly caused: This reinforces gender-based double standards and barriers for women in politics.

Inferred additional harm: Other female candidates globally may have faced similar gender-targeted deepfake harassment.

Civil rights

Reported: Yes, the right to free and fair elections and political participation was undermined.

Directly caused: Robocalls in the US urged voters to abstain from voting, directly targeting their civil right to vote.

Indirectly caused: Disinformation campaigns in Moldova and Slovakia sought to manipulate electoral outcomes, violating citizens' democratic rights.

Inferred additional harm: Large-scale voter suppression and manipulation likely occurred across multiple elections globally.

Democracy

Reported: Yes, the report explicitly states that deepfakes erode public trust in democratic institutions and electoral processes.

Directly caused: Deepfakes targeted key elections in Moldova, Slovakia, Taiwan, and the US to influence political outcomes.

Indirectly caused: The deluge of fake content erodes trust between people and in democratic institutions, making it harder for democracies to flourish.

Inferred additional harm: Long-term degradation of democratic stability and trust in public institutions globally.

Privacy

Reported: Yes, the unauthorized use of candidates' likenesses and voices.

Directly caused: The likenesses and voices of Maia Sandu, Joe Biden, and Rumeen Farhana were used without consent to create deepfakes.

Indirectly caused: N/A

Inferred additional harm: Thousands of individuals may have had their likenesses synthesized without consent for political or malicious purposes.

Psychological

Reported: The report explicitly describes public outrage and targeted smearing of candidates, which implies psychological distress.

Directly caused: Political candidates like Rumeen Farhana suffered reputational damage and distress due to sexualized deepfakes.

Indirectly caused: Voters experienced confusion and anxiety regarding the authenticity of election information.

Inferred additional harm: Other political figures targeted by deepfakes likely experienced significant stress, anxiety, and reputational distress.

Epistemic

Reported: Yes, the report describes the erosion of the public's trust in what they see and hear, blurring reality.

Directly caused: Fabricated audio of Slovakia's liberal leader and fake video of Moldova's president created false beliefs about their actions and statements.

Indirectly caused: A world in which everything is suspect makes it challenging for voters to make informed decisions based on shared facts.

Inferred additional harm: Widespread public cynicism and the 'liar's dividend' where politicians can dismiss real evidence as AI-generated.

People affected

• Occurrences reported: 5
• People reportedly harmed: 5
• People reportedly exposed: 100000000

Potential causes

Management

• Slow Response to AI Threat: Tech companies and governments fail to move fast enough to block fakes.

Technology

• Low-Cost Generative AI Tools: Free or cheap tools allow anyone to generate convincing deepfakes easily.
• Hard-to-Detect Audio Deepfakes: Manipulated audio lacks visual indicators, making detection very difficult.
• Misleading AI Chatbots: Chatbots generate false information that can disenfranchise voters.

Data Inputs

• Simple Text Prompts: Users can generate complex fakes using only basic text instructions.

Human Factors

• Low Media Literacy: Voters in some regions trust online content without verifying its source.
• Auditory Bias: Humans are less accustomed to detecting manipulation in audio than video.
• Malign Intent of Adversaries: Foreign actors use AI to deliberately manipulate elections and sow discord.

Process and Methods

• Easily Bypassed Safeguards: Users easily circumvent platform rules or use alternative services.
• Inadequate Detection Capabilities: Governments and tech companies lack the tools to quickly identify deepfakes.
• Encrypted Platform Distribution: Services like Telegram use encryption, making tracking and moderation hard.

Regulatory Environment

• Delayed Regulatory Mandates: Rules like EU's labeling requirements take effect too late for elections.
• Voluntary Tech Pacts: Non-binding agreements allow some platforms to avoid implementing safeguards.
• Lack of Global Standards: Global regulations are lagging behind the rapid advancement of AI tech.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, journalistic accounts of multiple specific deepfake incidents across different countries, citing experts, government officials, and targeted individuals.
• Ambiguities identified: The exact AI models used for each specific deepfake are not always identified, and the creators of the deepfakes are often anonymous or disputed.
• Alternative interpretations: Some might view these as standard political dirty tricks rather than a novel AI-driven threat, though the scale and ease of creation are uniquely AI-enabled.

The widespread deployment of generative AI to create political deepfakes represents a severe threat to information security and national sovereignty across multiple democracies. By targeting heads of state and electoral processes, these campaigns undermine the integrity of democratic transitions and erode public trust, requiring coordinated national and international responses.

• Overall national security impact: Severe
• Response level: Severe
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: United States, Moldova, Slovakia, Bangladesh, Taiwan
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Immediate. The threat is active and ongoing across multiple current global election cycles, requiring urgent response and continuous monitoring.
• Autonomy: Human-controlled. The AI systems function as tools operated directly by human actors who generate and disseminate the synthetic media.
• Novelty: Evolved capability. Represents a significant advancement in the scale, speed, and accessibility of existing digital disinformation and political sabotage methods.

Impact by dimension

• Physical security: Negligible. No physical security threats, kinetic actions, or critical infrastructure compromise were reported in relation to these incidents.
• Information security: Severe. Sophisticated, coordinated AI-generated deepfake campaigns targeted heads of state and political candidates to manipulate public perception and conduct information warfare.
• Sovereignty: Severe. The integrity of democratic electoral processes and sovereign government transitions was directly threatened by targeted AI-generated disinformation campaigns.
• Economic security: Negligible. The incident details do not indicate significant direct threats to economic stability, strategic technology theft, or financial systems.
• Societal stability: Substantial. The campaigns undermined public trust in democratic institutions, degraded shared epistemic reality, and involved targeted harassment of candidates.