Deepfake of Kamala Harris Saying Damaging Comments Circulates on X and Is Amplified by Elon Musk

The X user @MrReaganUSA uploaded a deepfake of Kamala Harris saying damaging comments about Joe Biden and her own qualifications for the presidency, originally marking it as a parody. The post was shared and amplified eight hours later via Elon Musk's account without the disclaimer.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale

The incident involves the use of AI-generated synthetic voice to mimic a political candidate, which was shared at scale to influence public opinion and the democratic process during an election campaign.

Additional risk subdomains

• 1.1 Unfair discrimination and misrepresentation: The video refers to Harris as a 'diversity hire' and mocks her race and gender, representing unfair misrepresentation.
• 3.1 False or misleading information: The video generates false and misleading audio of a public official making statements she never actually made.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was primarily caused by human actions, specifically the intentional creation of the altered video by a user and its subsequent intentional amplification by Elon Musk on X without appropriate labeling.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. The report describes an AI-generated deepfake video, which falls under the category of AI systems that pose a moderate risk and require specific transparency obligations, such as notifying users that the content is AI-generated.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Voice Generation; Deepfake Video Generation
• Behaviour type: Tool
• Alleged developer: Unknown deepfake technology developers
• Alleged deployer: X (Twitter), Elon Musk, @MrReaganUSA
• Alleged harmed parties: Truth, Kamala Harris, Joe Biden, General public, American voters

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Severe
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Minor, indirect Minor
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Severe
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes, the reports describe the creation and viral spread of a deepfake video containing misleading and derogatory statements.

Directly caused: An AI-manipulated video mimicking Kamala Harris's voice was created, calling her an incompetent candidate and a 'diversity hire'.

Indirectly caused: The video was amplified by Elon Musk, reaching over 119.9 million views on X, spreading derogatory and misleading content widely.

Differential treatment

Reported: Yes, the reports mention the video refers to Harris as a 'diversity hire' based on her race and gender.

Directly caused: The AI-generated voiceover explicitly targets Harris's gender and race, framing her selection as a 'diversity hire' and mocking her identity.

Democracy

Reported: Yes, the reports describe concerns from experts and politicians about the deepfake's potential to erode democratic discourse and influence voter behavior.

Directly caused: The dissemination of the unlabeled deepfake of a presidential candidate during an election campaign directly undermines the integrity of political communication.

Indirectly caused: The incident sparked widespread debate over platform policies and the lack of effective regulation for AI-generated campaign materials, potentially weakening public trust in democratic processes.

Epistemic

Reported: Yes, the reports describe the generation and viral spread of fabricated audio of a political figure.

Directly caused: The AI system generated highly realistic but entirely fabricated audio of Kamala Harris making self-deprecating and politically damaging statements.

Indirectly caused: The viral spread of the video without a parody label misled or confused viewers, eroding the shared consensus reality regarding what the candidate actually said.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 119900000

Potential causes

Management

• Prioritizing Engagement Over Safety: Platform owner prioritized viral engagement over enforcing safety guidelines.
• Lack of Executive Accountability: Leadership bypassed platform rules without facing internal enforcement actions.

Technology

• Generative AI Voice Cloning: AI tools allowed highly realistic manipulation of Harris's voice.
• Deepfake Creation Tools: Accessible technology enabled the creation of deceptive political video.

Data Inputs

• Original Campaign Video Audio: Unaltered campaign audio was manipulated to generate the fake voiceover.
• Lack of Digital Watermarks: Absence of metadata or watermarks allowed the video to circulate unlabeled.

Human Factors

• User Sharing Without Verification: Musk shared the deepfake to his 191M followers without verifying or labeling.
• Audience Susceptibility: Voters struggle to distinguish AI-generated content from reality.

Process and Methods

• Flawed Community Notes Process: X's crowdsourced moderation failed to append a warning note in a timely manner.
• Inconsistent Policy Enforcement: X failed to enforce its misleading media policy on a high-profile user.

Regulatory Environment

• Ambiguous Election Laws: The 1971 Federal Election Campaign Act is ambiguous regarding AI technology.
• Gridlocked Regulatory Bodies: FEC split along party lines, failing to pass rules on deceptive AI ads.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts of the incident, including the specific content of the deepfake, the platform where it was shared, the view counts, and the reactions from various stakeholders. The role of AI in generating the voice is clearly established, and the primary debate centers on platform policy enforcement rather than factual disputes about the event.
• Ambiguities identified: The specific AI tool used to generate the voiceover is not identified in the reports.

An AI-generated deepfake video of Vice President Kamala Harris went viral after being amplified by Elon Musk on X. The incident highlights the substantial threat of synthetic media to democratic elections and information security, emphasizing the challenges of platform policy enforcement and regulatory gaps regarding AI-generated political content.

• Overall national security impact: Substantial
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Elon Musk and X user @handle_001

Threat characteristics

• Imminence: Near-term. The threat of election-related deepfakes is an active, developing situation during the election cycle requiring close monitoring.
• Autonomy: Human-controlled. The AI system acted purely as a tool for voice generation directed by human creators and shared by human users.
• Novelty: Evolved capability. While deepfakes are an established threat, the scale of distribution and high-fidelity voice cloning represent an evolved capability.

Impact by dimension

• Physical security: Negligible. No physical security threat, kinetic action, or critical infrastructure compromise was reported in this incident.
• Information security: Substantial. AI-generated synthetic voice was used to impersonate a sitting Vice President, reaching over 119 million views on X, demonstrating potential for mass disinformation.
• Sovereignty: Substantial. The incident targeted a presidential candidate during an active election campaign, raising substantial concerns about the integrity of democratic processes.
• Economic security: Negligible. The incident did not involve financial system attacks, economic manipulation, or the theft of strategic AI technology.
• Societal stability: Minor. While the deepfake promoted derogatory framing based on race and gender, the societal impact was managed within standard political discourse.