In January 2024, a finance employee at the Hong Kong branch of the engineering firm Arup was defrauded of approximately $25 million USD. The employee was tricked into attending a video conference call where deepfake recreations of the company's CFO and other colleagues were present. Believing the call to be legitimate, the employee followed instructions to execute 15 wire transfers to five different bank accounts. The fraud was discovered a week later when the employee contacted the company's headquarters.
A finance employee at the multinational engineering firm Arup was reportedly deceived into transferring $25 million by fraudsters using purported deepfake technology to impersonate the firm's CFO in a video call, according to the Hong Kong police.
Risk classification
- Primary risk domain: 4 Malicious actors
- Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation
The scammers used AI deepfakes to impersonate the company's CFO and other colleagues on a video call to execute a highly targeted financial fraud scheme.
Additional risk subdomains
- 5.1 Overreliance and unsafe use: The employee over-relied on the visual and auditory realism of the deepfakes, overriding their initial correct suspicion of phishing.
Causal factors
- Entity: Human
- Intent: Intentional
- Timing: Post-deployment
The incident was caused by human fraudsters who intentionally used deployed deepfake AI tools to deceive an employee and steal corporate funds.
EU AI Act risk tier
- Risk tier: 3 Limited Risk
Limited Risk: The report describes the use of AI-generated deepfakes to impersonate individuals, which falls under the transparency obligations of Limited Risk systems.
AI system and alleged parties
- AI system: None named
- AI purpose: Deepfake Video Generation; Voice Generation
- Behaviour type: Tool
- Alleged developer: Unknown voice cloning technology developers, Unknown deepfake technology developers
- Alleged deployer: Unknown scammers
- Alleged harmed parties: Unnamed finance employee, Arup
Harm severity
Severity ratings are based on this scale.
Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.
- Physical: direct Negligible, indirect Negligible
- Infrastructure: direct Negligible, indirect Negligible
- Property: direct Negligible, indirect Negligible
- Financial: direct Substantial, indirect Negligible
- Environmental: direct Negligible, indirect Negligible
- Malicious content: direct Negligible, indirect Negligible
- Differential treatment: direct Negligible, indirect Negligible
- Civil rights: direct Negligible, indirect Negligible
- Democracy: direct Negligible, indirect Negligible
- Privacy: direct Negligible, indirect Negligible
- Psychological: direct Negligible, indirect Negligible
- Epistemic: direct Negligible, indirect Negligible
- Child sexual exploitation and abuse: direct Negligible, indirect Negligible
Financial
Reported: The reports explicitly describe a financial loss of approximately $25.6 million USD (HK$200 million) for the single occurrence.
Directly caused: Arup lost $25.6 million USD (HK$200 million) directly due to the unauthorized transfers executed during the scam.
Inferred additional harm: Potential additional costs for forensic investigation, legal fees, and implementing new security protocols, though not quantified.
People affected
- Occurrences reported: 1
- People reportedly harmed: 1
- People reportedly exposed: 4
Potential causes
Management
- Insufficient Security Training: Management did not update training to cover multi-person deepfake threats.
- Lack of Failsafe Procedures: No strict operational policies existed for highly confidential transactions.
Technology
- Advanced Deepfake Algorithms: Generative AI enabled highly realistic voice and video replication.
- Lack of AI Detection Tools: Collaboration platforms lacked real-time deepfake detection mechanisms.
- Vulnerability of Video Channels: Standard communication tools lack cryptographic key-pair verification.
Data Inputs
- Publicly Available Media: Scammers harvested executive video and audio from YouTube and public sources.
- Scripted Voice Training Data: Short public clips provided sufficient training data for voice cloning.
Human Factors
- Inherent Trust in Video: Employee assumed live video and recognized faces could not be faked.
- Cognitive Bias from Familiarity: Seeing recognized colleagues quelled the employee's initial doubts.
- Lack of Deepfake Awareness: Employee was untrained in identifying subtle AI-generation anomalies.
Process and Methods
- Single-Point Authorization: Lack of multi-level approvals allowed a single clerk to transfer millions.
- No Out-of-Band Verification: No protocol required verifying urgent requests via an alternative channel.
- Inadequate Transaction Monitoring: Fifteen anomalous transfers went undetected over a week-long period.
Regulatory Environment
- Lagging AI Legal Frameworks: Absence of strict regulations or liabilities for deepfake creation tools.
- Weak Platform Content Oversight: Social and video platforms lacked strict controls on executive impersonations.
Information quality
- Classification confidence: High
- Reason for confidence: The reports consistently detail the Arup incident, the amount stolen ($25.6 million USD / HK$200 million), the method used (multi-person deepfake video call), and the confirmation by Arup. There is very little ambiguity about the core facts of the heist.
- Ambiguities identified: Some minor discrepancies exist in early reports regarding whether other participants on the call were real or if the employee was the only real human, but later reports and police statements clarify the employee was the only real person.
A sophisticated financial fraud incident where scammers used multi-person deepfake video technology to impersonate corporate executives, resulting in a 25.6 million USD loss for engineering firm Arup. While representing a highly evolved technological threat capability, the national security impact remains minor as it targeted a private corporation for financial gain without disrupting critical infrastructure or state sovereignty.
- Overall national security impact: Minor
- Scope: Multiple nations
- Primary target: Hong Kong
- Other affected: United Kingdom
- Alleged perpetrator: Unknown
Threat characteristics
- Imminence: Long-term. The specific incident is resolved, but it represents an ongoing, long-term strategic threat of highly sophisticated AI-enabled fraud that organizations must defend against.
- Autonomy: Human-controlled. The AI deepfake tools were directly controlled and deployed by human fraudsters as tools to generate synthetic video and audio, without autonomous decision-making.
- Novelty: Evolved capability. Significant advancement in deepfake fraud, moving from single-person static impersonation to a live, multi-person video conference call simulation.
Impact by dimension
- Physical security: Negligible. The incident was purely financial fraud via deepfake video conferencing, with no physical systems, critical infrastructure, or human safety compromised.
- Information security: Minor. Sophisticated use of multi-person deepfake video to deceive, but targeted a private corporate entity for financial gain rather than state-level intelligence or systematic information warfare.
- Sovereignty: Negligible. The incident targeted a private engineering firm and did not impact government decision-making, electoral systems, or sovereign state operations.
- Economic security: Minor. Significant financial loss of 25.6 million USD to a major engineering firm, but manageable within corporate limits and does not threaten national economic stability.
- Societal stability: Negligible. The scam was a targeted corporate fraud event and did not cause widespread societal instability, civil unrest, or human rights violations.