Purported Deepfake TikTok Video Reportedly Featured Joe Rogan Endorsing Supplement Brand

A purported deepfake video featuring podcast host Joe Rogan reportedly advertising to his listeners about a "libido-boosting" supplement was circulating on TikTok and other platforms before being removed by TikTok along with the account which posted it.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involved the unauthorized use of Joe Rogan's public likeness and cloned voice to deceive consumers into purchasing a product for financial gain, which constitutes commercial fraud.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The scam was caused by human actors who intentionally used deployed AI voice-cloning tools to create a deceptive advertisement for financial gain.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The incident involves the creation of a deepfake video and voice clone, which falls under Risk Level 3 due to transparency obligations for synthetic media.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Voice Generation; Deepfake Video Generation
• Behaviour type: Tool
• Alleged developer: Unknown voice cloning technology developers, Unknown deepfake technology developers
• Alleged deployer: @mikesmithtrainer
• Alleged harmed parties: TikTok users, People seeking medical advice, Joe Rogan fans, Joe Rogan, General public, Epistemic integrity

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Minor
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Substantial
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes a malicious deepfake video ad created to scam users.

Directly caused: A deceptive deepfake video ad featuring a cloned voice of Joe Rogan was created and uploaded to TikTok to promote a product without his consent.

Indirectly caused: The video was reshared on Twitter, accumulating over 5 million views and spreading the deceptive content further.

Epistemic

Reported: The report explicitly describes epistemic harm through the creation of a fabricated conversation that misled viewers.

Directly caused: The AI-generated voice clone fabricated an endorsement of 'Alpha Grind' by Joe Rogan, misleading viewers into believing he supported the product.

Indirectly caused: The viral spread of the deepfake caused widespread concern and confusion among social media users about the authenticity of online videos.

Inferred additional harm: The proliferation of easily accessible voice-cloning tools is likely to cause widespread erosion of shared reality and trust in digital media.

People affected

• Occurrences reported: 1
• People reportedly harmed: 2
• People reportedly exposed: 5000000

Potential causes

Management

• Prioritizing Ad Revenue: Platforms prioritized ad volume over rigorous safety checks of promoted content.
• Inadequate Proactive Risk Plans: Platform management failed to prepare for rapid consumer-grade AI abuse.

Technology

• Accessible Voice Cloning Tools: Easy-to-use AI tools cloned celebrity voices with high accuracy in seconds.
• Advanced Cheapfake Techniques: Altering mouth movements and splicing audio made fake videos highly convincing.

Data Inputs

• Abundant Audio Training Data: Massive volumes of public podcast audio allowed precise voice model training.
• Unrelated Video Source Material: Real interview footages were easily harvested to splice with synthetic audio.

Human Factors

• User Susceptibility to Scams: Social media users struggled to distinguish the synthesized voice from reality.
• Malicious Intent by Scammers: Bad actors deliberately leveraged AI to deceive buyers and boost product sales.

Process and Methods

• Slow Content Moderation Systems: Platforms relied on manual reports rather than proactive automated AI detection.
• Inadequate Ad Screening Processes: TikTok approved and promoted the scam video as an ad without verification.

Regulatory Environment

• Slow Legislative Response: Federal regulations lack immediate protections against commercial deepfake scams.
• Lack of Industry Standards: No universal detection system or standard exists across AI software developers.

Information quality

• Classification confidence: High
• Reason for confidence: The reports consistently describe the same event: a deepfake ad of Joe Rogan promoting Alpha Grind on TikTok, which was confirmed as fake by his guest Andrew Huberman. The details regarding the platforms involved, the viral reach, and the removal of the content are clear and uncontradicted.
• Ambiguities identified: The exact AI tool used to clone the voice is not definitively confirmed, though ElevenLabs is mentioned as a likely candidate.

An AI-generated deepfake advertisement featuring cloned voices of public figures was used on TikTok for commercial fraud. While demonstrating the evolving sophistication and accessibility of synthetic media tools, the incident represents minor national security impact as it was limited to commercial scamming rather than state-sponsored information warfare.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding the proliferation of synthetic media tools rather than an active national security crisis.
• Autonomy: Human-controlled. The AI voice-cloning and video manipulation tools were directly operated and deployed by human actors for a specific fraudulent purpose.
• Novelty: Evolved capability. Reflects a significant advancement in the accessibility and realism of consumer-grade voice-cloning tools used to fabricate endorsements.

Impact by dimension

• Physical security: Negligible. No kinetic elements, threat to physical systems, or critical infrastructure compromise identified in this commercial advertisement incident.
• Information security: Minor. Demonstrates ease of creating highly realistic voice clones to mislead millions, but impact is minor as it was a commercial scam rather than a state-sponsored influence operation.
• Sovereignty: Negligible. No threat to state authority, electoral systems, or core government decision-making processes.
• Economic security: Minor. Represents minor economic impact through consumer fraud and unauthorized commercial exploitation of likenesses, but does not threaten strategic industries or financial systems.
• Societal stability: Negligible. While misleading to viewers, the incident does not threaten social cohesion, civil liberties, or cause population-scale instability.