Fake Biden Voice in Robocall Misleads New Hampshire Democratic Voters in 2024 Primary Election

A robocall imitating President Biden's voice urged New Hampshire Democrats to skip the 2024 primary, falsely claiming their votes mattered more in November. Investigators with the New Hampshire Attorney General's Office, along with other state AGs, the Industry Traceback Group, and the FCC determined that political consultant Steve Kramer hired Paul Carpenter to create the deepfake, which was distributed by Walter Monk's company, Life Corporation. Rep. Dean Phillips, who employed Kramer, has distanced himself from the incident. Kathy Sullivan, the former New Hampshire Democratic Party chair, was falsely impersonated in the caller ID data used for the robocalls.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale

The incident involved a coordinated campaign using AI-generated deepfake audio to spread disinformation at scale, targeting thousands of voters to influence a political process.

Additional risk subdomains

• 4.3 Fraud, scams, and targeted manipulation: The campaign relied on impersonating a trusted political figure (President Joe Biden) to manipulate the voting behavior of specific targets.
• 3.1 False or misleading information: The robocall generated and spread false information regarding voting rules, claiming that voting in the primary would invalidate votes in the general election.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was directly caused by the intentional actions of humans (Steve Kramer and Paul Carpenter) who used a deployed AI system to generate deceptive audio for a political campaign.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. The system was used to generate a deepfake, which falls under the category of AI-generated content requiring specific transparency obligations under the EU AI Act.

AI system and alleged parties

• AI system: ElevenLabs AI voice generator (ElevenLabs)
• AI purpose: Voice Generation
• Behaviour type: Tool
• Alleged developer: unknown
• Alleged deployer: unknown
• Alleged harmed parties: President Joe Biden, New Hampshire voters, Kathy Sullivan, Democracy

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Substantial
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Minor, indirect Minor
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: The report describes proposed regulatory fines of eight million dollars and operational costs incurred by advocacy groups.

Indirectly caused: The FCC proposed a six million dollar fine against Steve Kramer and a two million dollar fine against Lingo Telecom. Additionally, the League of Women Voters of New Hampshire incurred unquantified monetary costs to redirect resources toward voter education and guidance to counter the robocall.

Malicious content

Reported: The report explicitly describes the creation and distribution of a deceptive deepfake audio message impersonating President Joe Biden.

Directly caused: An AI-generated deepfake audio file mimicking Joe Biden's voice was created in under 20 minutes and distributed to thousands of voters.

Civil rights

Reported: The report explicitly describes the incident as an illegal attempt at voter suppression, which violates civil rights.

Directly caused: The robocall directly targeted thousands of Democratic voters with deceptive information in an attempt to suppress their right to vote in the primary election.

Inferred additional harm: It is possible that some voters were successfully deterred from voting, resulting in a direct deprivation of their civil rights, though the exact number is unconfirmed.

Democracy

Reported: The report explicitly describes the incident as an attack on democracy and an attempt to interfere with an election.

Directly caused: The deployment of AI-generated deepfakes to impersonate a sitting president and mislead voters represents a direct erosion of democratic norms and election integrity.

Indirectly caused: The incident forced regulatory bodies like the FCC and state attorneys general to rapidly implement new rules and launch extensive investigations to protect future elections.

Inferred additional harm: The success and ease of this operation may encourage other bad actors to deploy similar tactics, further eroding public trust in democratic processes and election outcomes.

Privacy

Reported: The report describes the spoofing of Kathy Sullivan's personal cellphone number.

Directly caused: The caller ID of the robocalls was spoofed to display Kathy Sullivan's personal cellphone number without her consent, violating her privacy.

Inferred additional harm: The personal phone numbers of thousands of voters were obtained from data brokers and targeted without their explicit consent, representing a broader privacy concern.

Psychological

Reported: The report describes Kathy Sullivan experiencing significant distress and disruption due to being bombarded with calls.

Indirectly caused: Kathy Sullivan was bombarded with a barrage of confrontational and confused phone calls after her personal number was spoofed, causing significant distress and disruption to her personal and professional life.

Inferred additional harm: It is likely that some vulnerable or elderly voters who received the call experienced temporary anxiety or confusion regarding their voting rights, though this is not quantified.

Epistemic

Reported: The report describes the generation of false information regarding voting rules using a deepfaked voice.

Directly caused: The AI-generated voice falsely told voters that voting in the primary would prevent them from voting in the November general election, undermining their understanding of the electoral system.

Inferred additional harm: The widespread media coverage of the deepfake may increase general public skepticism, making it harder for voters to distinguish between real and fake communications from political figures.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 5000

Potential causes

Management

• Lack of Campaign Oversight: The candidate's campaign failed to oversee independent contractor actions.
• Reactive Account Moderation: AI platform relied on banning accounts after the incident had occurred.

Technology

• Rapid Voice Cloning Tech: ElevenLabs cloned Biden's voice in under 20 minutes for only 1 dollar.
• Accessible Generative AI: Synthesizing highly realistic audio requires no specialized technical skills.

Data Inputs

• Abundant Public Voice Data: Decades of public speech data enabled highly accurate voice cloning.
• Voter Phone List Availability: Target phone numbers were easily purchased from commercial data brokers.

Human Factors

• Deceptive Intent of Consultant: Kramer deliberately planned the fake robocall to mislead voters.
• Uncritical Creator Compliance: Carpenter created the deepfake for money without verifying authorization.

Process and Methods

• Weak AI Platform Guardrails: ElevenLabs failed to prevent unauthorized voice cloning prior to creation.
• Lax Caller ID Authentication: Lingo Telecom gave high attestation without verifying caller ownership.

Regulatory Environment

• Lagging AI Legislation: Federal laws and Congress failed to keep pace with generative AI advancement.
• Vague Telecom KYC Standards: FCC lacked clear, enforceable standards for authenticating caller identities.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide highly detailed, consistent, and verified accounts of the incident, including admissions from the orchestrator (Steve Kramer) and the creator (Paul Carpenter), as well as official statements and legal filings from the FCC and the New Hampshire Attorney General.
• Ambiguities identified: The exact number of voters who received the call remains slightly ambiguous, with estimates ranging from just under 5,000 to 25,000.
• Alternative interpretations: Steve Kramer claimed the robocall was an act of civil disobedience to raise awareness about AI dangers, though prosecutors and plaintiffs interpreted it as a malicious attempt at voter suppression.

A political consultant commissioned an AI-generated deepfake audio robocall impersonating President Joe Biden to suppress voter turnout in the 2024 New Hampshire primary. This incident represents a notable escalation in AI-enabled election interference and information operations targeting democratic processes, though it was quickly identified and prosecuted.

• Overall national security impact: Substantial
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Steve Kramer

Threat characteristics

• Imminence: Long-term. The active incident has concluded with legal actions, but it represents an ongoing strategic concern for election integrity in future cycles.
• Autonomy: Human-controlled. The AI voice-cloning software was used strictly as a tool by human actors who generated, scripted, and distributed the audio.
• Novelty: Evolved capability. Represents a significant technological advancement in political impersonation and voter suppression, evolving from traditional robocalls to high-fidelity AI voice cloning.

Impact by dimension

• Physical security: Negligible. The incident was purely digital and did not target physical systems, critical infrastructure, or human safety.
• Information security: Substantial. Deployed an AI-generated deepfake voice of the sitting President to execute a targeted voter suppression campaign, representing a notable information operations incident.
• Sovereignty: Substantial. The incident targeted a state presidential primary election, attempting to manipulate the democratic process and suppress voter turnout through deceptive impersonation of the President.
• Economic security: Negligible. No significant impact on national economic or technological security, though regulatory bodies issued fines to the perpetrators and telecom carriers.
• Societal stability: Minor. Attempted voter suppression directly targeted the civil rights of several thousand voters, causing confusion but without leading to widespread civil unrest.