Purported AI-Generated Doctor Deepfakes Reportedly Used Guy's and St Thomas' Branding to Market Weight Loss Patches

Guy's and St Thomas' NHS Foundation Trust warned that purported AI-generated videos circulated on Facebook and TikTok depicted its clinicians endorsing weight loss patches. The videos allegedly impersonated doctors and used misleading medical claims to market a product.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves the use of AI-generated deepfake videos to impersonate NHS clinicians and deceive consumers into purchasing weight loss products for financial gain.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was caused by human scammers intentionally using deployed AI generation tools to create fraudulent promotional videos for financial gain.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The incident involves 'AI-generated content such as deepfakes' which, under the EU AI Act, are classified as limited risk and subject to specific transparency obligations to ensure users are informed of their synthetic nature.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Deepfake Video Generation; Social Media Content Generation
• Behaviour type: Tool
• Alleged developer: Unknown voice cloning technology developers, Unknown deepfake technology developers
• Alleged deployer: Unknown scammers
• Alleged harmed parties: People seeking medical advice, Guy's and St Thomas' NHS Foundation Trust clinicians, Guy's and St Thomas' NHS Foundation Trust, General public of the United Kingdom, General public, Epistemic integrity

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes

Directly caused: The creation and dissemination of AI-generated videos showing fake doctors endorsing weight loss products on Facebook and TikTok.

Inferred additional harm: It is likely that similar fraudulent videos using different fake personas were generated and distributed across the same or other social media platforms.

Epistemic

Reported: Yes

Directly caused: The dissemination of deceptive videos that falsely represent NHS clinicians as endorsing commercial weight loss patches, undermining public trust in medical professionals.

Inferred additional harm: The proliferation of such highly realistic deepfakes may lead to a broader erosion of public trust in official health advice and medical institutions.

People affected

• Occurrences reported: 1
• People reportedly exposed: 1000

Potential causes

Management

• Slow Takedown Response: Platform management was slow to respond to the Trust's takedown requests.

Technology

• Sophisticated AI Generation: AI tools allowed creating realistic videos of fake clinicians.
• Platform Detection Gaps: Social media platforms failed to automatically detect AI-generated scams.

Data Inputs

• Stolen Identity Data: Scammers used stolen online images, like bank ads, to create fake profiles.
• Unverified Promotional Content: Manipulated imagery and false claims were fed into social media feeds.

Human Factors

• Emotional Exploitation: Scammers bypassed rational thinking by targeting weight loss desires.
• Trust in Medical Authority: Public trust in NHS clinicians was leveraged to make the scam believable.

Process and Methods

• Reactive Moderation Processes: Platforms relied on manual reporting rather than proactive prevention.
• Lack of Profile Verification: Weak verification allowed fake doctor profiles with mismatched followers.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly describes the nature of the scam, the platforms involved, the reaction of the affected NHS Trust, and expert analysis of the fraudulent accounts. There is little ambiguity regarding the fact that AI-generated deepfakes were used for financial deception.
• Ambiguities identified: The specific AI tools or models used to generate the videos are not identified.
• Alternative interpretations: None. The event is clearly a commercial scam leveraging synthetic media.

An NHS Trust in London issued an alert regarding AI-generated deepfake videos of clinicians falsely endorsing weight loss patches. While the incident represents a malicious use of synthetic media to exploit public trust in state health services, it is a localized financial scam with negligible direct threat to national security.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United Kingdom
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Near-term. The active social media scam requires immediate monitoring and content takedowns, but does not pose a major national crisis.
• Autonomy: Human-controlled. AI was used as a generation tool by human actors who directed the content creation and dissemination.
• Novelty: Established threat. The use of AI-generated deepfakes for consumer-facing financial scams is an established threat method.

Impact by dimension

• Physical security: Negligible. No threats to physical infrastructure, kinetic operations, or physical safety of a critical nature occurred.
• Information security: Minor. Use of AI-generated deepfakes on social media represents a minor information threat, though it was localized to a financial scam rather than a state-sponsored influence campaign.
• Sovereignty: Minor. Impersonation of state-funded NHS clinicians slightly impacts public trust in government health institutions, but does not threaten state sovereignty or core government functions.
• Economic security: Negligible. Limited to a localized commercial scam targeting individual consumers with no systemic threat to financial systems or strategic economic security.
• Societal stability: Minor. The deepfakes create minor epistemic harm by undermining trust in medical advice, but do not threaten broad social cohesion or civil liberties.