Criminal Group Uses AI Deepfake Technology to Steal Personal Data in Hangzhou, Zhejiang

A criminal group in China used AI face-swapping technology to bypass face recognition systems on major platforms, steal personal data, and sell it to fraud syndicates. The group generated convincing video simulations from static photos to breach accounts, reportedly earning 200,000 yuan. After an investigation by the Hangzhou Public Security Bureau, four suspects were arrested across the provinces of Anhui, Guizhou, and Zhejiang.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The criminals used AI face-swapping technology to impersonate users, bypass security, and steal personal data for financial gain, which fits targeted exploitation and fraud.

Additional risk subdomains

• 2.1 Compromise of privacy by obtaining, leaking or correctly inferring sensitive information: The AI-generated videos were directly used to breach user accounts and illegally obtain sensitive personal information.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The risk and subsequent harm were caused by the intentional actions of human criminals who utilized a deployed AI model to bypass security systems.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the use of an AI model to generate face-swapping videos (deepfakes), which falls under transparency obligations for AI-generated content.

AI system and alleged parties

• AI system: 多模态专用大模型
• AI purpose: Deepfake Video Generation; Face Recognition
• Behaviour type: Tool
• Alleged developer: Unknown deepfake detection technology developers
• Alleged deployer: Zhang Mouguo, Wu Mouhao, Hu Mouyun, Hu Mouliang
• Alleged harmed parties: Zhejiang citizens, Guizhou citizens, Chinese citizens, Anhui citizens

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Minor, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: The report explicitly describes financial gains by the criminals of nearly 200,000 yuan, which represents a corresponding financial loss/theft from the platforms or users.

Directly caused: The criminal gang profited nearly 200,000 yuan from their illicit activities, representing direct financial loss/theft of asset value from the victims.

Inferred additional harm: The stolen personal information sold to fraud syndicates likely led to much larger downstream financial losses for the victims of those scams, potentially totaling millions of yuan.

Civil rights

Reported: The report explicitly describes the violation of citizens' personal information rights.

Directly caused: The criminals violated citizens' personal information rights by stealing their photos, generating fake videos, and selling their private data.

Privacy

Reported: The report explicitly describes severe privacy violations where personal information was stolen and sold.

Directly caused: The criminals bypassed facial recognition to log into accounts, steal personal information, and sell it to fraud syndicates.

Inferred additional harm: The downstream use of this stolen personal data by fraud syndicates likely resulted in further severe privacy violations and targeted scams against the victims.

People affected

• Occurrences reported: 1
• People reportedly harmed: 100
• People reportedly exposed: 100

Potential causes

Management

• Slow Security Upgrades: Platform management delayed upgrading authentication against AI threats.

Technology

• Multimodal AI Generation: Overseas AI models generated realistic face-liveness videos from photos.
• Vulnerable Face Authentication: Platform login systems failed to distinguish real faces from AI videos.

Data Inputs

• Stolen Static Photos: Static photos were stolen and used as input data for the AI model.

Human Factors

• Poor Device Security: Users clicked unsafe links, compromising their personal information.
• Insider Misconduct: Employees leaked personal data, giving attackers targets for AI bypass.

Process and Methods

• Weak Verification Process: Platforms lacked optical liveness checks to block deepfake videos.

Regulatory Environment

• Unregulated Overseas AI: Lack of controls on overseas AI models enabled malicious deepfakes.

Information quality

• Classification confidence: High
• Reason for confidence: The police report provides clear, factual details about the crime, the technology used (multimodal large model for face-swapping), the suspects arrested, and the financial scale of the operation (200,000 yuan). There is little ambiguity about the events.
• Ambiguities identified: The exact number of victims whose accounts were compromised is not specified, and the specific platforms targeted are only referred to as 'head platforms'.
• Alternative interpretations: None. The incident is clearly a cybercrime leveraging generative AI for identity theft and fraud.

A Chinese criminal group utilized an offshore multimodal AI model to generate deepfake videos, successfully bypassing facial recognition authentication on major platforms to steal and sell personal data. While the incident highlights evolving vulnerabilities in biometric security, the impact remains a localized cybercrime with minor national security implications, successfully resolved by local law enforcement.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: China
• Alleged perpetrator: Criminal group

Threat characteristics

• Imminence: Long-term. The specific criminal group has been arrested, representing an ongoing strategic capability concern for biometric security rather than an active crisis.
• Autonomy: Human-controlled. The AI system acted purely as a tool, generating specific video frames based on manual text and image inputs from the criminals.
• Novelty: Evolved capability. Represents an evolution of existing deepfake and presentation attack techniques using newer multimodal large models to bypass biometric security.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or human safety threats were involved in this digital fraud incident.
• Information security: Minor. Demonstrates capability to bypass facial recognition security systems, but used for commercial fraud rather than state-sponsored intelligence or information warfare operations.
• Sovereignty: Negligible. Standard criminal activity handled by local law enforcement with no disruption to state authority or government functions.
• Economic security: Minor. Financial theft of approximately 200,000 yuan and compromise of personal data, representing minor economic impact managed by standard law enforcement.
• Societal stability: Minor. Compromise of personal data and privacy violations of platform users, but limited in scale and without systemic threats to social cohesion or human rights.