Rep. Mike Collins's Campaign Allegedly Produced Deepfake of Sen. Jon Ossoff Supporting the Government Shutdown

Rep. Mike Collins's congressional campaign allegedly created and posted an AI-generated video that purportedly used Sen. Jon Ossoff's portrait and synthetic voice to falsely depict him supporting the ongoing government shutdown. The video was reportedly shared on the campaign's X account with a limited disclaimer. Georgia Democrats criticized the deepfake as misleading, and the campaign reportedly indicated it would continue using similar AI tactics.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale

The campaign used generative AI to create a deepfake video of a political opponent to influence public opinion and manipulate the political process during an election.

Additional risk subdomains

• 3.1 False or misleading information: The AI-generated video fabricated statements that the Senator never made, spreading false information to the electorate.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was caused by humans (the political campaign) intentionally using a deployed generative AI tool to create and distribute a deceptive deepfake video of an opponent.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the creation of AI-generated content such as deepfakes, which are subject to transparency obligations (disclosures) under the Act.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: Unknown voice cloning technology developers, Unknown deepfake technology developers
• Alleged deployer: Mike Collins's campaign, Mike Collins
• Alleged harmed parties: Truth, Jon Ossoff, General public of Georgia, General public, Epistemic integrity, Electoral integrity, Democracy

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Minor, indirect Minor
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes, the report describes the creation and spread of a deepfake video containing fabricated statements.

Directly caused: The campaign directly created and posted a deepfake video on social media featuring computer-generated audio of Ossoff claiming to support a government shutdown.

Inferred additional harm: It is likely that the video was reshared by other social media users, expanding its reach to tens of thousands of voters.

Democracy

Reported: Yes, the report explicitly describes concerns about the use of AI to mislead voters and manipulate elections.

Directly caused: The deepfake video directly targeted a U.S. Senate race, fabricating a candidate's positions to influence voters.

Indirectly caused: The incident sparked a wider debate over the responsible use of emerging technologies in political persuasion and the potential for deepfakes to erode trust in elections.

Inferred additional harm: Widespread use of such deepfakes could lead to systemic voter confusion and a decline in overall voter turnout and trust in democratic institutions.

Epistemic

Reported: Yes, the report describes the generation of false information and the blurring of truth and fabrication.

Directly caused: The campaign used synthetic audio to make Ossoff falsely claim he supported the shutdown and had 'only seen a farm on Instagram'.

Indirectly caused: A voter was reported as 'incredulous' and confused about whether the video was authentic, demonstrating the erosion of shared truth.

Inferred additional harm: Further epistemic harm is highly likely as voters struggle to distinguish real campaign communications from AI-generated fabrications.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 10000

Potential causes

Management

• Prioritizing Strategy over Truth: Campaign management prioritized winning elections over factual accuracy.
• Lack of Campaign Accountability: Campaign leadership defended deceptive AI use as legitimate satire.

Technology

• Realistic Voice Cloning: AI tools generated highly realistic audio mimicking Ossoff's voice.
• Lack of Media Provenance: AI-generated media lacks traceable origin, making source verification hard.

Data Inputs

• Official Portraits as Source Data: Official Senate portrait was used to create a realistic visual deepfake.
• Scripted Fabricated Statements: Campaign-scripted text was used to generate fake audio statements.

Human Factors

• Voter Cognitive Bias: People hear fabricated audio once and assume it is true.
• Intentional Misleading Tactics: Campaigns intentionally use deepfakes to manipulate voter perceptions.

Process and Methods

• Obscured Disclaimer Design: AI disclaimer was small and hidden behind mobile screen captions.
• Viral Social Media Distribution: Manipulated videos are easily shared online, reaching many voters quickly.

Regulatory Environment

• Absence of Clear Federal Laws: Lack of nationwide regulations allows deceptive AI campaign materials.
• State Legislative Failures: Proposed state bills to regulate deepfakes died in the Senate.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, consistent details about the creation, content, and distribution of the deepfake video, as well as the reactions from both campaigns and experts. The role of AI is explicitly stated, and the primary harms (epistemic and democratic) are well-documented.
• Ambiguities identified: The exact AI tools used to generate the video and voice are not specified.
• Alternative interpretations: The campaign argued the video was legitimate political satire rather than a deceptive deepfake, though this does not change its classification as AI-generated political content.

A domestic political campaign in Georgia utilized generative AI to create a deceptive deepfake video of Senator Jon Ossoff. While the incident highlights the growing threat of synthetic media to democratic norms and voter trust, its national security impact remains minor as it was conducted by domestic actors within a localized election context.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: U.S. political campaign

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding the integration of generative AI in political campaigns rather than an active national security crisis.
• Autonomy: Human-controlled. The AI system was used strictly as a tool by human operators who scripted the content and generated the media.
• Novelty: Established threat. The use of deepfakes and synthetic audio to target political candidates is an established threat with numerous prior examples.

Impact by dimension

• Physical security: Negligible. No physical systems, infrastructure, or human safety elements were affected or targeted in this incident.
• Information security: Minor. Represents AI-enhanced political disinformation, but conducted by domestic political actors rather than a foreign intelligence operation.
• Sovereignty: Minor. Involves deceptive tactics in a U.S. Senate election, potentially affecting democratic processes, but managed within existing legal and political frameworks.
• Economic security: Negligible. No strategic technology theft, financial system impact, or critical supply chain disruption occurred.
• Societal stability: Minor. Contributes to the erosion of epistemic trust and voter confusion, but does not involve mass surveillance, systematic discrimination, or civil unrest.