Many Purported Political Deepfakes Reportedly Circulating in Run-up to 2024 Pakistani General Elections

During Pakistan's 2024 general elections, purportedly AI-generated deepfakes of a politically motivated nature were circulated. These deepfakes reportedly portrayed political figures in misleading contexts, spreading misinformation and aiming to influence voter perceptions and election outcomes.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale

The incident involved coordinated, politically motivated deployment of AI-generated deepfakes to influence public opinion and manipulate the Pakistani general elections.

Additional risk subdomains

• 3.1 False or misleading information: The AI systems generated highly realistic but entirely false audio and video representations of political figures to mislead voters.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The risk was caused by human actors intentionally using deployed AI deepfake generation tools to create and spread political disinformation during an election.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. The report describes the creation and dissemination of 'AI-generated completely new videos or audio', which are explicitly defined as deepfakes under the Act and subject to transparency obligations.

AI system and alleged parties

• AI system: None named
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: Unknown voice cloning technology developers, Unknown deepfake technology developers
• Alleged deployer: Pakistani political parties, Misinformation spreaders, Misinformation networks, Disinformation spreaders
• Alleged harmed parties: Truth, Rana Atif, Raja Bashara, Naeem Haider Panjutha, Imran Khan, General public of Pakistan, General public, Epsitemic integrity, Electoral integrity, Democracy

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Minor
• Democracy: direct Minor, indirect Minor
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes malicious content created and spread directly by the incident in the form of political deepfakes.

Directly caused: AI-generated deepfake videos and audio of political candidates (Raja Bashara, Imran Khan, Naeem Haider Panjutha, Rana Atif) were created and spread on social media platforms like X, Facebook, and TikTok.

Inferred additional harm: It is likely that additional unmentioned deepfakes and politically motivated synthetic media were generated and circulated during the election cycle.

Civil rights

Reported: The report describes concerns from global institutions regarding the election environment, which indirectly relates to the civil right to participate in free and fair elections.

Indirectly caused: The dissemination of fake videos urging boycotts or party changes interfered with the democratic rights of voters to receive accurate information.

Democracy

Reported: The report explicitly describes the erosion of democratic norms and the manipulation of public discourse caused directly by the deepfakes.

Directly caused: Deepfakes of key political figures falsely announced election boycotts and party defections, directly undermining the integrity of the electoral process.

Indirectly caused: The circulation of these fakes contributed to general confusion and distrust among the electorate during a critical democratic event.

Inferred additional harm: The widespread use of such technology likely caused long-term erosion of trust in democratic institutions and official communications in Pakistan.

Epistemic

Reported: The report explicitly describes epistemic harm, noting the 'erosion of truth baseline on social media' and the spread of 'fake news, disinformation, and misinformation'.

Directly caused: The AI-generated deepfakes fabricated statements from politicians, creating false evidence of political decisions (boycotts, defections) that did not occur.

Indirectly caused: Candidates were forced to release urgent clarification videos to correct the record and re-establish the truth.

Inferred additional harm: Widespread confusion among voters who may have believed the deepfakes and acted upon false information, permanently distorting their perception of the candidates' stances.

People affected

• Occurrences reported: 4
• People reportedly harmed: 4
• People reportedly exposed: 4

Potential causes

Technology

• Deep Learning Algorithms: Algorithms learn from data to generate highly realistic fake media.
• AI Video and Audio Synthesis: AI-generated media convinces viewers that someone said or did something.

Data Inputs

• Target Training Data: Deep learning models require source data to learn and mimic targets.

Human Factors

• Political Disinformation Motivations: Actors knowingly spread untrue AI content to influence voters.
• Public Deception Vulnerability: Unnatural details in deepfakes are difficult for average users to detect.

Process and Methods

• Rapid Social Media Viral Spread: Deepfakes circulate quickly on platforms like X, Facebook, and WhatsApp.
• Reactive Clarification Process: Candidates must manually post videos to debunk AI disinformation.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides specific details about multiple distinct deepfake incidents targeting named political figures during the 2024 Pakistani elections, including their responses and clarifications. The role of AI in generating these deepfakes is explicitly stated, and the political context is well-documented.
• Ambiguities identified: The specific AI models or platforms used to generate the deepfakes are not identified.
• Alternative interpretations: None. The events are clearly described as politically motivated disinformation using synthetic media.

During the 2024 Pakistani general elections, multiple AI-generated deepfakes of political candidates were circulated to spread disinformation, including false announcements of boycotts and defections. This incident represents a substantial threat to information security and sovereignty by targeting democratic processes, demonstrating an evolved capability of human-controlled AI tools to manipulate public discourse at scale.

• Overall national security impact: Substantial
• Scope: Single nation
• Primary target: Pakistan
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. While the active election crisis has passed, the strategic threat of AI-driven election interference remains an ongoing concern for national security.
• Autonomy: Human-controlled. The AI deepfake generators were utilized as tools by human operators who directed the creation and dissemination of the specific disinformation content.
• Novelty: Evolved capability. While election disinformation is established, the coordinated deployment of multiple deepfakes targeting key candidates represents an evolved capability.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or kinetic targets were impacted or threatened during this incident.
• Information security: Substantial. Coordinated deployment of synthetic media targeting key political figures to manipulate voter perception during a national election represents a notable information warfare effort.
• Sovereignty: Substantial. The incident directly targeted the electoral process, a core government function, by fabricating statements about party defections and election boycotts.
• Economic security: Negligible. No significant economic systems, strategic technologies, or financial infrastructures were targeted or impacted.
• Societal stability: Minor. The disinformation created epistemic confusion and eroded public trust in democratic processes, though it did not trigger large-scale civil unrest.