Purported Deepfake of Ashley James Reportedly Used to Promote Weight Loss Pills

An online ad reportedly used a purported AI-generated version of Ashley James, a British broadcaster and former reality television personality, to market weight loss pills through a false celebrity endorsement. James said the video copied her face and voice without consent and described the ad as both a violation of her identity and a misleading sales tactic directed at the public.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves the unauthorized use of a public figure's likeness and voice clone to create a fake endorsement for financial gain through a weight loss pill scam.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was caused by malicious human actors intentionally deploying a deepfake video to scam the public using a post-deployment AI tool.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system involves AI-generated content such as deepfakes, which pose a moderate risk and require transparency obligations to ensure users are informed of the AI's nature.

AI system and alleged parties

• AI system: None named
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: Unknown voice cloning technology developers, Unknown deepfake technology developers
• Alleged deployer: Unknown scammers, Unknown advertisers
• Alleged harmed parties: People seeking weight loss supplements, Fans of Ashley James, Ashley James

Harm severity

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes, the report describes an AI-generated deepfake video advertisement used maliciously to deceive the public.

Directly caused: An incredibly realistic AI-generated video of [PERSON_002] was created and spread on social media, falsely depicting her endorsing weight loss pills.

Indirectly caused: N/A

Inferred additional harm: The malicious ad was likely viewed by thousands of social media users before being reported or taken down.

Psychological

Reported: The report explicitly describes psychological distress experienced by [PERSON_002] due to the incident.

Directly caused: [PERSON_002] was left 'devastated' and felt a 'total violation' because her likeness was used to promote weight loss pills, which directly contradicts her years of body-positive advocacy.

Indirectly caused: N/A

Inferred additional harm: Members of the public who bought the pills believing her recommendation may have experienced distress, anxiety, or body image issues upon realizing they were scammed.

Epistemic

Reported: Yes, the report describes epistemic harm through the creation of a highly realistic deepfake that misled viewers into believing [PERSON_002] endorsed the product.

Directly caused: The AI ad fabricated a scenario where [PERSON_002] claimed she lost 27 pounds using the pills, eroding the shared truth of her actual beliefs and public statements.

Indirectly caused: N/A

Inferred additional harm: The proliferation of such realistic deepfakes erodes public trust in video evidence and authentic celebrity endorsements on social media.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 1000

Potential causes

Management

• Platform Profit Over Safety: Social media companies prioritized ad revenue over rigorous safety checks.

Technology

• Realistic Voice and Image Synthesis: AI tools allowed highly realistic replication of the presenter's likeness.
• Deepfake Generation Tools: Accessible AI technology enabled the creation of convincing fake advertisements.

Data Inputs

• Unauthorized Likeness Training Data: Publicly available video and audio of the presenter were used without consent.

Human Factors

• Targeting of Trusting Audiences: Scammers exploited public trust in the presenter to sell weight loss products.
• User Susceptibility to Deepfakes: The AI ad was realistic enough that viewers assumed it was genuine.

Process and Methods

• Inadequate Platform Moderation: Social media platforms allowed the distribution of unverified deepfake ads.
• Flawed Ad Review Procedures: Platform ad approval processes failed to detect the fraudulent deepfake content.

Regulatory Environment

• Lack of Deepfake Legal Recourse: Unclear legal frameworks make it difficult to stop or penalize deepfake ads.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly details the nature of the incident, the victim's response, and the medium used (AI-generated video and voice). There is no ambiguity about the fraudulent nature of the ad or the impact on the victim.
• Ambiguities identified: The specific AI tools or platforms used to generate the deepfake are not identified.

An AI-generated deepfake of a British broadcaster was used in an online weight loss scam. While representing a violation of individual identity and a tool for commercial fraud, the incident has negligible national security implications and does not target critical infrastructure, government functions, or state sovereignty.

• Overall national security impact: Minor
• Response level: Moderate
• Scope: Single nation
• Primary target: United Kingdom
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding commercial deepfakes rather than an active crisis requiring urgent national security response.
• Autonomy: Human-controlled. The AI system functioned strictly as a generation tool under direct human control to create the deepfake video and voice clone.
• Novelty: Established threat. Deepfake technology used for celebrity-endorsed commercial scams is a well-established and frequently occurring threat vector.

Impact by dimension

• Physical security: Negligible. No physical threat to critical infrastructure, weapons systems, or public safety was identified in this commercial scam incident.
• Information security: Minor. Involves AI-generated deepfake media to deceive the public, but is limited to a commercial fraud campaign rather than a state-sponsored information warfare operation.
• Sovereignty: Negligible. The incident does not impact state authority, electoral processes, or core government decision-making.
• Economic security: Negligible. While causing minor individual financial fraud, the incident does not threaten national financial systems or critical technological sectors.
• Societal stability: Minor. Represents an identity violation of a public figure and misleads the public, but does not threaten overall social cohesion or civil liberties.