Reported Deepfake Influencers on TikTok Allegedly Used to Promote Fraudulent Wellness Products

A network of TikTok accounts reportedly used AI-generated and deepfake influencer personas to promote wellness and beauty products through alleged deceptive advertising. The accounts reportedly invented identities and personal testimonials while making unsubstantiated medical claims, including false assertions related to fertility and hair-growth treatments. Some personas were reportedly reused across multiple accounts with inconsistent backstories. The content purportedly targeted vulnerable users and linked to unverified products before the accounts were removed.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves scammers using AI-generated deepfake influencers and fabricated stories to deceive users into purchasing unverified wellness products for financial gain.

Additional risk subdomains

• 3.1 False or misleading information: The deepfake influencers and slideshows spread unverified and false health claims regarding PCOS treatments and hair growth.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was caused by human scammers who intentionally deployed AI-generated deepfake influencers post-deployment to deceive TikTok users and sell wellness products.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the use of 'AI-generated content such as deepfakes' which falls under Limited Risk due to transparency obligations to inform users they are interacting with AI-generated media.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Deepfake Video Generation; Deepfake Image Generation
• Behaviour type: Tool
• Alleged developer: Unknown voice cloning technology developers, Unknown generative AI developers, Unknown deepfake technology developers
• Alleged deployer: Unknown TikTok account operators, Unknown scammers
• Alleged harmed parties: TikTok users, People seeking medical advice, General public, Epistemic integrity, Consumers of wellness and beauty products

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes toxic or malicious content created or spread directly by the incident.

Directly caused: The scammers directly created and uploaded numerous TikTok videos featuring AI-generated deepfake influencers and deceptive slideshows to promote wellness products.

Inferred additional harm: It is likely that thousands of TikTok users were exposed to these malicious, deceptive deepfake videos before the accounts were removed.

Epistemic

Reported: The report explicitly describes epistemic harm caused directly by the incident.

Directly caused: The AI-generated deepfake influencers fabricated elaborate personal stories and false medical claims, such as claiming batana oil increases hair growth and Hey Girl supplements manage PCOS.

Inferred additional harm: The widespread dissemination of these deepfakes contributes to the erosion of public trust in online health information and medical professionals.

People affected

• Occurrences reported: 1

Potential causes

Management

• Reactive Moderation Policies: TikTok only removed the scam accounts after media reports exposed them.

Technology

• Sophisticated Deepfake Video: AI tools enable creation of highly realistic talking deepfake influencers.
• Generative AI Image Creation: AI generates realistic images of people for fake storytime slideshows.

Data Inputs

• Unlabeled AI Content: Deepfake media lacks clear labels indicating it is AI-generated.
• Stolen Likeness Data: Using real people's images to anchor or generate deepfake personas.

Human Factors

• Trust in Wellness Buzzwords: Users are easily misled by health claims and familiar story formats.
• Exploitation of Vulnerabilities: Scammers target women with fertility issues using emotional stories.

Process and Methods

• Weak Automated Detection: TikTok's automated filters failed to flag coordinated deepfake networks.
• Coordinated Account Networks: Scammers easily deploy the same AI content across multiple accounts.

Regulatory Environment

• Inadequate AI Ad Regulations: Lack of strict laws governing AI-generated influencers in advertising.

Information quality

• Classification confidence: High
• Reason for confidence: The report from Media Matters provides clear, documented examples of specific TikTok accounts, their deceptive strategies, and the types of AI-generated content used. While the exact AI models are not named, the nature of the deepfake technology and its application in the scam is highly transparent.
• Ambiguities identified: The specific AI tools or platforms used by the scammers to generate the deepfakes are not identified.

A network of TikTok accounts utilized AI-generated deepfake influencers to execute deceptive commercial wellness scams targeting vulnerable individuals. While representing an evolution in consumer fraud sophistication, the incident has negligible national security implications and does not target critical infrastructure, government functions, or state stability.

• Overall national security impact: Minor
• Scope: Unknown
• Primary target: No clear primary
• Other affected: Unknown
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The specific accounts have been removed, making this an ongoing strategic concern regarding deepfake scam proliferation rather than an active crisis.
• Autonomy: Human-controlled. AI was utilized strictly as a generation tool by human actors who directed the creation and deployment of the deceptive content.
• Novelty: Evolved capability. Represents a clear evolution in online scam tactics, upgrading from simple stock images to sophisticated talking deepfake personas.

Impact by dimension

• Physical security: Negligible. No threat to physical systems, critical infrastructure, or physical safety of a national security nature.
• Information security: Negligible. The incident involves commercial consumer fraud rather than state-sponsored disinformation, espionage, or intelligence compromise.
• Sovereignty: Negligible. No impact on state authority, electoral systems, or core government operations.
• Economic security: Negligible. The financial impact is limited to individual consumer scam victims and does not threaten strategic industries, financial systems, or national technological advantages.
• Societal stability: Minor. Deceptive targeting of medically vulnerable populations (women with PCOS and fertility issues) with false health claims represents a minor threat to public trust, but lacks the scale to threaten broad societal stability.