Scammers Reportedly Used Deepfake Videos to Blackmail a TikTok User Posting About Bhutan

A TikTok user who posted about missing Bhutan while living abroad reportedly faced blackmail involving manipulated videos described as deepfakes. Scammers allegedly sent the videos to some of his contacts after he initially dismissed their threats. The episode reportedly caused distress. The AI tool and his country of residence were not identified.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The primary incidents involve using AI deepfakes to impersonate public figures for commercial promotion without consent, and targeting individuals for blackmail and harassment.

Additional risk subdomains

• 1.2 Exposure to toxic content: The report mentions non-consensual intimate image-based abuse (NCII), which constitutes exposure to toxic and sexually degrading synthetic content.
• 6.3 Economic and cultural devaluation of human effort: The unauthorized replication of the likeness and voices of famous singers and influencers devalues their personal brand and creative identity.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The deepfakes were intentionally created and posted by human actors (the startup founders and scammers) using post-deployment open-source AI models.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the creation and use of 'AI-generated content such as deepfakes' and 'Chatbots', which are classified as Limited Risk under the EU AI Act, requiring transparency obligations.

AI system and alleged parties

• AI system: LLaMA2, Falcon 7b
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: Synthetic video generation technology developers, Synthetic media generation technology developers, Deepfake technology developers
• Alleged deployer: Synthetic media creators, Scammers, Extortionists, Deepfake creators
• Alleged harmed parties: Victims of extortion, Victims of deepfake abuse, TikTok users, Sonumbtopguy, Social media users, Privacy, Epistemic integrity

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Substantial, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Minor, indirect Minor
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the creation and spread of unauthorized deepfakes, blackmail videos, and intimate images.

Directly caused: Scammers created and sent manipulated blackmail videos of Sonumbtopguy to his contacts, and NoMindBhutan created unauthorized promotional deepfakes of three public figures.

Inferred additional harm: Additional unquantified deepfake videos and non-consensual intimate images are likely circulating on social media platforms in Bhutan.

Civil rights

Reported: The report explicitly describes violations of personal rights, including consent and protection from harassment.

Directly caused: The right to control one's own likeness was violated for the three public figures, and the personal dignity of the blackmail and NCII victims was violated.

Privacy

Reported: The report explicitly describes privacy violations through the non-consensual use of personal likenesses and data.

Directly caused: NoMindBhutan used the faces and voices of three individuals without consent, and scammers harvested Facebook details to target Sonumbtopguy.

Psychological

Reported: The report explicitly describes psychological harm, including trauma and distress, caused by the deepfake scams.

Directly caused: TikTok user Thukten Wangmo Gyeltshen described her experience with deepfakes as 'traumatizing', and Sonumbtopguy described his blackmail experience as 'distressing'.

Indirectly caused: The three public figures expressed concern and fear regarding the potential misuse of their likenesses.

Inferred additional harm: It is highly likely that other unnamed victims of deepfake blackmail and non-consensual intimate image abuse in Bhutan experienced similar psychological trauma and anxiety.

Epistemic

Reported: The report explicitly describes epistemic harm, noting that deepfakes blur the line between reality and deception.

Directly caused: The unauthorized deepfakes of public figures misled or confused viewers regarding their actual endorsement of NoMindBhutan.

Indirectly caused: The rise of deepfakes has eroded general trust in online media, forcing consumers to become highly skeptical of everything they see online.

People affected

• Occurrences reported: 3
• People reportedly harmed: 5
• People reportedly exposed: 1000

Potential causes

Management

• Prioritizing Growth Over Ethics: Startup prioritized quick deployment over ethical considerations.
• Inadequate Risk Management: Management failed to assess the risks of publishing unconsented deepfakes.

Technology

• Accessible Open-Source AI Models: Easy access to deepfake models via online platforms like Hugging Face.
• Advanced Voice and Face Cloning: AI models can replicate faces and voices with high realism and ease.

Data Inputs

• Unprotected Social Media Media: Publicly available images and videos are harvested to train deepfakes.
• Lack of Consent for Personal Data: Using influencers' faces and voices without obtaining explicit consent.

Human Factors

• Disregard for Explicit Consent: Creators assumed social media likes on comments equaled consent.
• Lack of Online Skepticism: Users trust malicious links and online content too easily.

Process and Methods

• Weak Verification Processes: No formal process to verify consent before publishing AI content.
• Insecure Link Clicking: Clicking unknown links led to account freezing and data harvesting.

Regulatory Environment

• Lack of AI Governance laws: Absence of clear regulations on deepfakes and non-consensual media.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly details the specific deepfake incidents, the parties involved, and the resulting psychological and privacy harms. The classification of deepfakes under the EU AI Act and the Causal/Domain taxonomies is straightforward based on the provided text.
• Ambiguities identified: The report combines a commercial dispute over unauthorized promotional deepfakes with criminal blackmail and intimate image abuse scams under a general discussion of deepfake risks.
• Alternative interpretations: The NoMindBhutan incident could be viewed purely as an intellectual property or consent dispute rather than a malicious scam, but the inclusion of blackmail and NCII cases in the report justifies the broader 'Malicious actors' classification.

An AI startup in Bhutan used unauthorized deepfakes of public figures for promotion, while other citizens faced deepfake blackmail and non-consensual intimate image abuse. These incidents highlight growing localized societal risks and privacy violations from accessible AI tools, but carry negligible direct threats to national security, critical infrastructure, or state sovereignty.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: Bhutan
• Alleged perpetrator: NoMindBhutan and unnamed scammers

Threat characteristics

• Imminence: Long-term. Represents an ongoing societal and regulatory challenge rather than an active national security crisis.
• Autonomy: Human-controlled. AI tools were operated locally by human developers and scammers to generate specific deepfake content.
• Novelty: Established threat. Deepfakes used for unauthorized advertising, blackmail, and harassment are well-established global threat vectors.

Impact by dimension

• Physical security: Negligible. No kinetic threats, physical attacks, or critical infrastructure disruptions were reported in connection with these deepfake incidents.
• Information security: Minor. Involves unauthorized deepfakes of public figures and personal blackmail, but lacks indicators of state-sponsored information warfare or intelligence compromise.
• Sovereignty: Negligible. No disruption to government functions, electoral systems, or state sovereignty was reported.
• Economic security: Negligible. No strategic technology theft or financial system attacks occurred; impacts are limited to localized commercial unauthorized use.
• Societal stability: Minor. Erosion of individual privacy and human rights through non-consensual intimate imagery and blackmail, though localized and not at a mass population scale.