Purportedly AI-Generated Deepfake Ads on Facebook Reportedly Impersonate Trump, Musk, Ocasio-Cortez, Warren, Sanders, and Leavitt to Promote Fraudulent Rebates

A Tech Transparency Project investigation identified purportedly AI-generated deepfake ads on Facebook impersonating President Trump, Elon Musk, Rep. Alexandria Ocasio-Cortez, Senators Elizabeth Warren and Bernie Sanders, and Press Secretary Karoline Leavitt. The ads allegedly promoted fake $5,000 government rebates and similar scams, reportedly misleading users and generating ad revenue for Meta before removal.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves malicious actors using AI-generated deepfakes of public figures to conduct targeted financial scams and fraud against seniors.

Additional risk subdomains

• 1.2 Exposure to toxic content: The AI-generated deepfakes were used to promote illegal financial scams, exposing users to deceptive and harmful content.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was caused by human scammers intentionally using deployed generative AI tools to create deepfakes for financial fraud.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The incident involves the creation and dissemination of AI-generated content, specifically deepfakes of political figures, which falls under the transparency obligations of this level.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: Unknown voice cloning technology developers, Unknown deepfake technology developers, Meta
• Alleged deployer: Unknown scammers operating Meta ad accounts, Unknown scammers, Senior Health Daily USA (advertiser), RFY News Group (advertiser network), Relief Eligibility Center (advertiser), Meta, I Love My Freedom (advertiser), Health Benefits for Seniors (advertiser), Hannah Grace (advertiser), Get Covered Today (advertiser), Freedom Asset Advocates (advertiser), End the Wokeness (advertiser), Asking America (advertiser), American Relief Programs (scam network)
• Alleged harmed parties: Public trust, Meta users, Karoline Leavitt, Instagram users, General public, Facebook users, Epistemic integrity, Elon Musk, Elizabeth Warren, Elderly individuals, Donald Trump, Bernie Sanders, Alexandria Ocasio-Cortez

Harm severity

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Substantial, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: Yes. The report describes scammers spending $49 million on ads to facilitate financial fraud. It notes individual victims losing significant amounts, such as one 76-year-old losing $443.84, and the FTC reporting older adults losing $10,000 or more, sometimes their entire life savings.

Directly caused: The average financial loss per occurrence is difficult to calculate directly, but the 63 scam advertisers spent $49 million on ads, implying massive direct financial extraction from victims.

Indirectly caused: N/A

Inferred additional harm: Given the $49 million ad spend and millions of users reached, total inferred financial losses to victims likely range in the tens of millions of dollars.

Malicious content

Reported: Yes. The report describes the creation and distribution of deepfake videos of political figures used to deceive users.

Directly caused: Over 150,000 deceptive ads featuring deepfakes of Donald Trump, Bernie Sanders, Elizabeth Warren, and others were distributed on Facebook and Instagram, reaching millions of users.

Indirectly caused: N/A

Inferred additional harm: It is likely that many more undetected AI-generated scam ads were distributed across Meta's platforms, exposing millions of additional users to malicious content.

Differential treatment

Reported: Yes. The report explicitly states that the scam ads targeted seniors and older adults.

Directly caused: Scammers specifically targeted men and women over the age of 65 with ads promoting fake Medicare benefits and stimulus checks, exploiting their vulnerability.

Indirectly caused: N/A

Inferred additional harm: Seniors were disproportionately targeted and harmed by these AI-enabled scams compared to other demographic groups.

Privacy

Reported: Yes. The report describes ads directing users to websites that harvest personal information.

Directly caused: Ads prompted users to enter their names, email addresses, and credit card details on fraudulent websites under the guise of claiming benefits or free merchandise.

Indirectly caused: N/A

Inferred additional harm: Thousands of users likely had their personally identifiable information and financial data harvested and potentially sold or exploited further.

Epistemic

Reported: Yes. The report describes the fabrication of statements by public figures using deepfake technology.

Directly caused: Deepfake videos fabricated statements from Donald Trump, Bernie Sanders, Elizabeth Warren, and Karoline Leavitt, falsely claiming the government was issuing stimulus checks or dividend cards.

Indirectly caused: N/A

Inferred additional harm: The proliferation of highly realistic deepfakes of public figures contributes to the erosion of shared truth and increases public skepticism toward authentic video evidence.

People affected

• Occurrences reported: 1
• People reportedly harmed: 145
• People reportedly exposed: 1000000

Potential causes

Management

• Prioritizing Revenue Over Safety: Meta focused on short-term ad revenue over long-term platform safety.
• Reduced Platform Safety Teams: Meta rolled back moderation efforts and downsized safety divisions.
• Inadequate Advertiser Vetting: Management failed to implement rigorous verification for political ads.

Technology

• Generative AI Deepfakes: Scammers used AI to create realistic fake videos of politicians.
• Automated Ad Approval System: Meta relied on automated checks that failed to detect deepfake scams.
• Lack of Alteration Detection: Platform lacked technical tools to automatically identify manipulated media.

Data Inputs

• Unverified Advertiser Credentials: Advertisers submitted unverified or foreign IDs to bypass political checks.
• Missing Digital Disclosures: Scam ads lacked mandatory disclosures for digitally altered content.
• Deceptive Landing Page Data: Ads linked to external sites that gathered user info and credit cards.

Human Factors

• Targeting of Vulnerable Seniors: Scammers exploited older adults using trusted political figures.
• Public Confusion on Benefits: Confusion around social safety net programs made scams more believable.
• User Trust in Political Figures: Victims trusted the deepfaked politicians appearing in the video ads.

Process and Methods

• Lax Ad Moderation Process: Meta allowed scam ads to run for days or weeks before taking action.
• Excessive Account Strike Policy: Meta allowed up to 32 automated strikes before banning scammer accounts.
• Crowdsourced Enforcement Model: Meta waited for user reports instead of proactively blocking scam ads.

Regulatory Environment

• Outdated AI Laws: Laws and consumer protections for rapid AI advances are outdated.
• Platform Liability Disclaimers: Meta argued in court that it does not owe a duty to protect users.
• Blocked Subscription Rules: Court rulings blocked FTC efforts to crack down on negative option scams.

Information quality

• Classification confidence: High
• Reason for confidence: The reports from the Tech Transparency Project, The New York Times, and other sources provide detailed, consistent, and quantified evidence of the scam campaigns, including ad spend, platforms used, specific politicians deepfaked, and the targeting of seniors. The role of generative AI in creating the deepfakes is clearly established.
• Ambiguities identified: The exact financial losses suffered specifically by the victims of these 63 advertisers are not fully quantified, only general FTC trends and individual BBB complaints are cited.
• Alternative interpretations: None. The evidence clearly points to intentional financial fraud using AI-generated deepfakes.

Scammers utilized generative AI to create highly realistic deepfakes of prominent U.S. political figures to run a massive, 49 million dollar fraudulent ad campaign targeting seniors on Meta platforms. While the incident represents a significant evolution in AI-enabled financial fraud and public deception, its national security impact remains minor as it does not target critical infrastructure, state sovereignty, or national economic systems.

• Overall national security impact: Minor
• Response level: Moderate
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing, persistent capability concern regarding AI-enabled fraud rather than an immediate national security crisis.
• Autonomy: Human-controlled. Generative AI tools were used by human actors to create content, with all campaign decisions and targeting executed manually by the scammers.
• Novelty: Evolved capability. Represents a significant advancement of existing impersonation and phishing tactics through the integration of highly realistic generative AI deepfakes.

Impact by dimension

• Physical security: Negligible. No physical threats, kinetic attacks, or critical infrastructure disruptions were reported in connection with this incident.
• Information security: Minor. While deepfakes of prominent political figures were deployed at scale, the operation was designed for financial fraud rather than state-sponsored geopolitical influence or intelligence compromise.
• Sovereignty: Minor. Impersonation of federal officials to promote fake government programs causes public confusion regarding official communications but does not disrupt core government operations.
• Economic security: Minor. The incident involved millions of dollars in fraudulent losses for individuals, but does not threaten national financial systems or strategic economic sectors.
• Societal stability: Minor. Targeted exploitation of elderly citizens causes localized financial and psychological harm, eroding trust in digital media without causing systemic civil unrest.