Deepfake Obama Introduction of Deepfakes

University of Washington researchers made a deepfake of Obama, followed by Jordan Peele

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The primary risk demonstrated is the generation of highly realistic, fabricated video and audio content that can mislead audiences and undermine the credibility of visual evidence.

Additional risk subdomains

• 4.1 Disinformation, surveillance, and influence at scale: The reports highlight the severe potential for deepfakes to be weaponized in coordinated political disinformation campaigns to manipulate public opinion and elections.
• 4.3 Fraud, scams, and targeted manipulation: The underlying technology was widely exploited to create non-consensual deepfake pornography targeting specific female celebrities.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The creation of the deepfake videos by Jordan Peele and the development of the synthesis software by researchers were intentional human actions utilizing deployed AI tools.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The technology described is used to generate deepfakes and synthetic media. Under the EU AI Act, AI-generated content such as deepfakes is classified as Limited Risk (Level 3), requiring transparency obligations so that users are aware the content is AI-generated.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: University of Washington, FakeApp
• Alleged deployer: University of Washington, FakeApp
• Alleged harmed parties: Barack Obama

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Substantial, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Negligible, indirect Minor
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Substantial
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes, the reports describe the creation and spread of non-consensual deepfake pornography on platforms like Reddit.

Directly caused: The FakeApp tool was used to generate non-consensual pornographic videos of female celebrities by mapping their faces onto porn stars' bodies.

Inferred additional harm: It is highly likely that numerous other individuals, particularly women, have been targeted with non-consensual deepfake pornography that was not explicitly detailed in these reports.

Civil rights

Reported: Yes, the reports discuss the violation of personal rights through non-consensual deepfake pornography.

Directly caused: The unauthorized mapping of female celebrities' faces onto pornographic content violated their personal rights and bodily autonomy.

Inferred additional harm: Many non-celebrity individuals likely had their civil rights and privacy violated through the creation of revenge porn using these widely available tools.

Democracy

Reported: Yes, the reports express significant concern over the potential use of deepfakes to influence elections and erode trust in democratic institutions.

Indirectly caused: The proliferation of these tools has contributed to a general erosion of public trust in video evidence and media authenticity.

Inferred additional harm: The availability of these tools likely facilitates ongoing covert disinformation campaigns aimed at undermining democratic institutions globally.

Privacy

Reported: Yes, the reports mention the unauthorized use of individuals' likenesses and voices to train AI models.

Directly caused: The likenesses of female celebrities were used without consent to generate pornographic videos.

Inferred additional harm: Thousands of individuals likely have had their public photos and videos harvested without consent to train personalized deepfake models.

Epistemic

Reported: Yes, the reports explicitly describe the creation of fabricated videos that undermine the reliability of visual evidence.

Directly caused: The creation of the realistic fake video of Barack Obama speaking fabricated sentences directly demonstrated the ability to generate convincing false evidence.

Indirectly caused: The existence of such technology creates a 'liar's dividend' where public figures can claim genuine footage of their misconduct is fake, eroding the shared consensus reality.

Inferred additional harm: Widespread public skepticism towards all digital media is likely increasing, leading to a broader loss of consensus reality.

People affected

• Occurrences reported: 2
• People reportedly harmed: 5
• People reportedly exposed: 4800000

Potential causes

Management

• Academic Tech Release: Researchers publish powerful tools without fully assessing societal risks.
• Corporate Funding of AI Research: Tech giants fund AI research prioritizing capabilities over safety tools.
• Lack of Platform Accountability: Social networks fail to implement real-time fake detection systems.

Technology

• Open-Source AI Algorithms: Public release of FakeApp and open-source deep learning code on Reddit.
• Advanced Neural Networks: Recurrent neural networks and GANs enable highly realistic lip-syncing.
• Voice Cloning Tech: Software like Lyrebird replicates voices from short audio samples.

Data Inputs

• Abundant Public Video Footage: Hours of high-definition footage of public figures are freely available.
• Lack of Input Verification: Neural nets are trained on unverified internet data without restrictions.
• Social Media Image Uploads: Users upload thousands of tagged images, increasing exposure to fakes.

Human Factors

• Confirmation Bias: People tend to believe fake videos that align with their political views.
• Low Media Literacy: Unsuspecting viewers struggle to identify subtle digital manipulations.
• Malicious Intent of Actors: Pornographers and fraudsters exploit the tech for revenge porn and scams.

Process and Methods

• Inadequate Detection Tools: Reverse engineering AI to detect fakes lags behind generation methods.
• Lack of Watermarking Protocols: Absence of standard digital signatures or watermarks on edited media.
• Tedious Manual Verification: Rigorous newsroom verification processes are slow to counter viral fakes.

Regulatory Environment

• Absence of Global Protocols: No international standards exist to authenticate digital media identity.
• Delayed Government Policies: Technological evolution rapidly outpaces the development of policy safeguards.
• Inadequate Platform Bans: Banning users on Reddit or other sites fails to stop technology spread.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts of the development of lip-syncing AI by the University of Washington and the creation of the Obama deepfake PSA by Jordan Peele and BuzzFeed. The technical mechanisms (RNNs, GANs, FakeApp) and the societal implications are clearly articulated across multiple reputable sources.
• Ambiguities identified: There is some ambiguity regarding the exact number of individuals affected by the non-consensual deepfake pornography mentioned, as the reports only refer to 'female celebrities' generally, with Gal Gadot being the only one explicitly named.
• Alternative interpretations: The incident could be interpreted purely as a benign scientific breakthrough in video editing rather than an AI safety incident, though the widespread concern over misinformation and non-consensual porn supports the safety risk classification.

The public emergence of realistic AI-generated deepfakes, demonstrated by the University of Washington's lip-syncing research and the Jordan Peele Obama PSA, established a significant new capability for information warfare and societal manipulation, prompting immediate defense research responses like DARPA's media forensics program.

• Overall national security impact: Substantial
• Scope: Multiple nations
• Primary target: United States
• Alleged perpetrator: Individuals

Threat characteristics

• Imminence: Long-term. The proliferation of deepfake technology represents an ongoing, strategic threat to information integrity rather than an immediate, time-sensitive crisis.
• Autonomy: Human-controlled. The AI tools (FakeApp, RNNs) require direct human input, training data, and execution, acting as tools rather than autonomous agents.
• Novelty: First-of-its-kind. This incident marked the public emergence of photorealistic AI-generated deepfakes of global leaders, establishing a novel precedent for synthetic media threats.

Impact by dimension

• Physical security: Negligible. No physical security or critical infrastructure threats were reported in connection with this incident.
• Information security: Substantial. The creation of realistic deepfakes of former US President Obama demonstrated a new capability for information warfare, prompting DARPA to initiate media forensics programs to counter potential disinformation.
• Sovereignty: Minor. While raising concerns for future electoral manipulation, the actual incident was a domestic public service announcement and academic research, representing minor immediate impact on government functions.
• Economic security: Minor. Represents a technological advancement in generative AI, but did not involve strategic theft, financial attacks, or direct economic warfare.
• Societal stability: Substantial. The technology was immediately weaponized for non-consensual pornography targeting individuals and threatened epistemic security by eroding public trust in visual media.