Purportedly AI-Generated Audio Allegedly Fabricates Biden's Admission of Role in Pakistani Political Crisis

A purported deepfake audio file is alleged to have falsely claimed that U.S. President Joe Biden conspired with Pakistan's Army Chief, General Syed Asim Munir, to remove former Prime Minister Imran Khan in 2022. Widely shared online, the audio is reported to have exploited generative AI to spread political misinformation.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale

The AI-generated audio was deployed as part of a coordinated effort to influence public opinion and political processes at scale.

Additional risk subdomains

• 3.1 False or misleading information: The AI system was used to generate false and misleading information that fabricated a statement by a world leader.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was caused by a human actor who intentionally generated and distributed the deepfake audio to mislead the public.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes an AI-generated deepfake. Under the EU AI Act, AI systems that generate or manipulate image, audio or video content (deepfakes) pose a moderate risk and are subject to specific transparency obligations.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Voice Generation; Text Style Replication
• Behaviour type: Tool
• Alleged developer: Unknown deepfake technology developers
• Alleged deployer: Unknown deepfake creators
• Alleged harmed parties: Syed Asim Munir, Relations between the U.S. and Pakistan, People of Pakistan, Joe Biden, Imran Khan

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Minor
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Minor, indirect Minor
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes

Directly caused: The incident directly involved the creation of a malicious deepfake audio designed to impersonate US President Joe Biden.

Indirectly caused: The deepfake was widely shared across multiple social media platforms, exposing tens of thousands of users to malicious political content.

Democracy

Reported: Yes

Directly caused: The deepfake directly targeted political figures and diplomatic relations by fabricating a conspiracy confession.

Indirectly caused: The spread of the fake audio among political supporters could undermine trust in democratic processes and international relations.

Epistemic

Reported: Yes

Directly caused: The AI-generated audio fabricated a confession by Joe Biden, creating false evidence of a geopolitical conspiracy.

Indirectly caused: The fake audio misled social media users, polluting the information ecosystem during sensitive election periods.

People affected

• Occurrences reported: 1
• People reportedly harmed: 2
• People reportedly exposed: 34100

Potential causes

Technology

• Generative AI Voice Cloning: AI algorithms cloned Biden's voice to create realistic but fake statements.
• Splicing and Crossfading: Combining audio snippets and adding music to mask synthetic artifacts.
• Absence of Natural Speech Pauses: AI failed to replicate natural breathing spaces and human speech cadences.

Data Inputs

• Use of Public Voice Samples: Abundant public recordings of Joe Biden were used to train the voice model.
• Poor Grammatical Training Data: Linguistic errors in the script resulted in ungrammatical synthetic speech.
• Background Music Integration: Adding background music to the input audio file masked AI-generation traces.

Human Factors

• Political Polarization: Supporters eagerly shared the fake audio to validate conspiracy narratives.
• Lack of Public Verification: Users forwarded the audio without verifying its authenticity or source.
• Intentional Misinformation: Creators timed the release to influence the 2024 US and Pakistani politics.

Process and Methods

• Inadequate Platform Moderation: Social media platforms failed to flag or block the viral deepfake audio.
• Lack of Proactive AI Detection: No automated tools were used by networks to detect synthetic voice clones.
• Sidelined Fact Checking: Fact-checking occurred only after the deepfake had achieved viral reach.

Regulatory Environment

• Lack of AI Laws in Pakistan: No existing legal framework or penalties governed the malicious use of AI.
• Delayed Regulatory Legislation: The draft AI regulation bill was only recently introduced in the Senate.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides detailed forensic and linguistic analyses from multiple independent experts confirming the audio is AI-generated. The dissemination platforms and view counts are well-documented.
• Ambiguities identified: The specific AI tool used to generate the voice is not identified.
• Alternative interpretations: None. The consensus among experts is highly consistent that the audio is synthetic.

An AI-generated deepfake audio of US President Joe Biden falsely claiming a conspiracy with the Pakistani military was circulated on social media. The incident represents a substantial information warfare concern, exploiting generative AI to impact international relations and domestic political narratives in multiple nations, though it caused no physical or critical infrastructure damage.

• Overall national security impact: Substantial
• Scope: Multiple nations
• Primary target: Pakistan
• Other affected: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. This incident represents an ongoing strategic concern regarding the proliferation of synthetic media in political discourse rather than an active, immediate crisis.
• Autonomy: Human-controlled. The AI voice cloning system was utilized strictly as a tool by a human actor to generate specific content, without autonomous decision-making.
• Novelty: Established threat. AI-generated deepfakes of political figures have occurred previously, making this an instance of an established threat vector rather than a novel capability.

Impact by dimension

• Physical security: Negligible. No physical security threats, kinetic attacks, or critical infrastructure compromise were reported in relation to this incident.
• Information security: Substantial. The incident involved a highly sophisticated AI-generated deepfake of the US President to propagate a false geopolitical conspiracy, representing a notable information warfare effort to manipulate public opinion.
• Sovereignty: Minor. While the deepfake falsely implicated the US government in foreign regime change, it did not lead to direct compromise of core government decision-making processes or sovereign control.
• Economic security: Negligible. No threats to financial systems, strategic technologies, or economic stability were observed or reported.
• Societal stability: Minor. The circulation of the deepfake among political supporters in a polarized environment contributes to epistemic harm and could exacerbate social polarization, though no civil unrest was reported.