Scammers Deepfaked Videos of Victims' Loved Ones Asking for Funds over Facebook in Vietnam

In Vietnam, to convince victims of their disguises when prompted, scammers deepfaked audios and videos of victims' friends and families asking them over Facebook to send over thousands of dollars.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The primary reported harm is the targeted exploitation of specific victims for financial gain using deepfake video calls to impersonate trusted individuals.

Additional risk subdomains

• 6.3 Economic and cultural devaluation of human effort: Celebrities like MC Quyen Linh and Dr. Truong Huu Khanh had their public likenesses and reputations exploited without permission for unauthorized commercial advertisements.
• 4.1 Disinformation, surveillance, and influence at scale: Deepfake technology was also reported to be used by anti-state channels to spread political propaganda and fabricate stories about Vietnam's socio-economic status.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The harms were caused by human scammers and malicious actors intentionally utilizing deployed deepfake technology to commit financial fraud and spread unauthorized advertisements.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3. Limited Risk: The report describes the creation and use of AI-generated content such as deepfakes, which require transparency obligations under the Act so users are aware they are interacting with synthetic media.

AI system and alleged parties

• AI system: Deepfake
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: unknown
• Alleged deployer: Unknown scammers
• Alleged harmed parties: Vietnamese Facebook users

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Minor
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Democracy

Reported: Yes

Indirectly caused: Deepfake technology was used on anti-state channels to spread political propaganda, fabricate stories about Vietnam's socio-economic status, and go against the Party and State.

Inferred additional harm: The ongoing use of deepfakes to target political figures and spread propaganda could further erode public trust in democratic institutions and state governance.

Privacy

Reported: Yes

Directly caused: Scammers harvested visual and personal data of victims from social media platforms like Facebook and Zalo to train and generate the deepfakes.

Inferred additional harm: Thousands of social media users are at risk of having their public photos and videos harvested without consent to create deepfakes.

Psychological

Reported: Yes

Directly caused: N.C.P. experienced panic and distress upon realizing he was scammed of 100 million VND. MC Quyen Linh and Dr. Truong Huu Khanh experienced ongoing frustration and annoyance over their likenesses being used in unauthorized medical ads.

Inferred additional harm: Many other unnamed victims likely experienced significant distress, anxiety, and panic after losing large sums of money to these highly convincing deepfake scams.

Epistemic

Reported: Yes

Directly caused: Deepfake video calls fabricated the visual and vocal presence of trusted individuals, misleading victims into believing they were talking to their relatives.

Indirectly caused: Deepfakes of celebrities and doctors fabricated endorsements for medicines, misleading the public into buying unapproved treatments.

Inferred additional harm: The proliferation of deepfakes degrades the overall information ecosystem in Vietnam, making it difficult for citizens to distinguish between real and synthetic media.

People affected

• Occurrences reported: 2
• People reportedly harmed: 4
• People reportedly exposed: 6

Potential causes

Management

• Weak Ad Verification: Ad networks accept spam advertisements without verifying permits.
• Slow Response to Hacked Accounts: Platforms take too long to freeze and secure hacked user accounts.

Technology

• Deepfake Synthesis Tools: Easy online access to tools that generate realistic video and audio.
• Low-Quality Call Masking: Scammers hide deepfake flaws by mimicking weak network connections.

Data Inputs

• Public Media Harvesting: Scammers collect personal photos and videos from Facebook and Zalo.
• Personal Data Scraping: Scammers gather contact lists, phone numbers, and relationships.
• Voice Recording Exploitation: Scammers call victims to record their voice and intonation patterns.

Human Factors

• Overreliance on Video Calls: Victims trust blurry video calls without secondary verification.
• Urgency and Panic Exploitation: Scammers exploit victim emotions regarding financial emergencies.
• Lack of Deepfake Awareness: General public struggle to distinguish between real and deepfaked media.

Process and Methods

• Account Takeover via Hacking: Compromising social media accounts to initiate trusted conversations.
• Cross-Border Coordination: Scammers operate across international borders, complicating response.
• Unauthorized Image Use: Using celebrity images without consent to market unapproved products.

Regulatory Environment

• Lack of Ad Enforcement: Agencies struggle to track and punish unauthorized virtual ads.
• Inadequate Platform Regulation: Social media platforms lack robust controls to prevent deepfake spreads.
• Jurisdictional Challenges: Cross-border cybercrimes lack unified international legal enforcement.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, detailed accounts of specific victims (N.C.P. and C.T.N.Y.), the amounts of money lost, the methods used by the scammers, and the perspectives of affected public figures. The role of deepfake technology is explicitly stated.
• Ambiguities identified: The specific deepfake software or platforms used by the scammers are not identified.
• Alternative interpretations: None. The events are clearly characterized as cyber-enabled financial fraud and unauthorized likeness exploitation.

Deepfake technology is being deployed in Vietnam for financial fraud, unauthorized commercial endorsements, and political propaganda by anti-state channels. While primarily a cyber-enabled fraud issue, the use of synthetic media for political propaganda represents a substantial information warfare risk to Vietnam's informational security.

• Overall national security impact: Substantial
• Scope: Single nation
• Primary target: Vietnam
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The threat represents an ongoing strategic and capability concern regarding the proliferation of synthetic media tools, rather than an immediate crisis.
• Autonomy: Human-controlled. The AI tools are manually operated by human scammers to generate synthetic content, with no autonomous decision-making or execution by the AI system itself.
• Novelty: Evolved capability. This represents a significant advancement of existing impersonation and phishing tactics by integrating sophisticated synthetic video and voice generation.

Impact by dimension

• Physical security: Negligible. No kinetic threats, physical harm, or critical infrastructure compromise were reported in connection with this incident.
• Information security: Substantial. Deepfake technology was actively utilized by anti-state channels to spread political propaganda and fabricate stories about Vietnam's socio-economic status, targeting the information ecosystem.
• Sovereignty: Minor. While anti-state channels spread propaganda against the state, there was no direct disruption of core government operations, electoral systems, or constitutional processes.
• Economic security: Minor. The financial losses were restricted to individual fraud cases and unauthorized commercial endorsements, posing no systemic risk to Vietnam's national economic stability.
• Societal stability: Minor. The scams caused psychological distress and financial harm to specific individuals, and eroded trust in digital media, but did not trigger widespread civil unrest or systemic oppression.