Voice deepfake targets bank in failed transfer scam

In spring 2023, Florida investor Clive Kabatznik became the target of an advanced scam attempt involving a voice deepfake mimicking his own voice. The fraudulent caller, using AI-generated speech, contacted Kabatznik's Bank of America representative in an unsuccessful attempt to deceive the banker into transferring funds to a different account.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves scammers using AI-generated voice cloning to impersonate a specific individual in order to execute a targeted financial scam against a bank.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was caused by human scammers intentionally using a post-deployment AI voice cloning tool to attempt financial fraud.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system is used to generate AI-generated content such as deepfakes, which falls under Risk Level 3 due to transparency obligations to ensure users are aware they are interacting with AI-generated content.

AI system and alleged parties

• AI system: VALL-E, unspecified (Microsoft)
• AI purpose: Voice Generation; Text Style Replication
• Behaviour type: Tool
• Alleged developer: unknown
• Alleged deployer: Unknown scammers
• Alleged harmed parties: Clive Kabatznik, Bank of America

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Psychological

Reported: The report explicitly describes psychological distress experienced by the targeted investor.

Directly caused: The investor experienced distress and anxiety upon realizing his voice had been cloned to target his bank account, describing the situation as 'pretty scary' and wondering if he should 'go underground and disappear'.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 2

Potential causes

Management

• Narrow Security Focus: Securing call centers while leaving direct mobile lines vulnerable.

Technology

• Generative AI Voice Cloning: AI programs clone voices using very short audio samples.
• Low-Cost Generative AI Tools: Cheap tools make sophisticated voice synthesis widely accessible.
• Text-to-Speech Software: Type-to-text programs let attackers type what the synthetic voice says.

Data Inputs

• Public Voice Recordings: Speeches and social media videos provide ample audio for cloning.
• Stolen Customer Data: Stolen bank details sold on underground markets help target victims.

Human Factors

• Susceptibility to Voice Spoofing: Employees can be tricked by realistic cloned voices of known clients.

Process and Methods

• Caller ID Spoofing: Attackers easily spoof target numbers to bypass caller ID trust.
• Inadequate Direct Verification: Lack of robust out-of-band authentication for direct phone calls.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides clear, firsthand details of the specific scam attempt targeting the investor, including the banker's reaction, the subsequent communication lockout, and the source of the audio data used. The role of AI voice cloning is explicitly stated and well-documented.

An unknown attacker used generative AI voice cloning to impersonate a Florida investor in a failed attempt to execute a fraudulent bank transfer. While the scam was unsuccessful, it demonstrates the evolving capability of malicious actors using AI for targeted financial fraud, carrying minor national security implications primarily around economic security and synthetic media risks.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding generative AI-enabled financial fraud rather than an active crisis requiring immediate response.
• Autonomy: Human-controlled. The AI voice cloning tool was directly controlled and utilized by human scammers to generate the synthetic speech.
• Novelty: Evolved capability. Significant advancement of traditional voice impersonation scams utilizing sophisticated generative AI voice cloning technology.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or human safety was threatened or affected by this incident.
• Information security: Minor. Involves AI-generated synthetic media (voice deepfake) targeting an individual, but lacks the scale, coordination, or intelligence compromise of state-level information warfare.
• Sovereignty: Negligible. No impact on government operations, electoral systems, state authority, or constitutional processes.
• Economic security: Minor. Targeted financial fraud attempt on a commercial bank account. No strategic assets or systemic financial infrastructure were compromised, resulting in zero direct financial loss.
• Societal stability: Minor. Caused psychological distress and a 10-day communication lockout for the targeted individual, but had no broader impact on societal stability or human rights.