Canadian Parents Tricked out of Thousands Using Their Son's AI Voice

Two Canadian residents were scammed by an anonymous caller who used AI voice synthesis to replicate their son's voice asking them for legal fees, disguising as his lawyer.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involved scammers using AI voice-cloning technology to impersonate trusted family members for direct financial gain through fraud.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The financial fraud was intentionally executed by human scammers utilizing a deployed AI voice-cloning tool to deceive victims.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. The reports describe 'deepfakes and other AI-based synthetic media' and 'AI voice-generating software' used to clone voices, which falls under the category of AI-generated content such as deepfakes requiring transparency obligations.

AI system and alleged parties

• AI system: VALL-E, VoiceLab, unspecified (ElevenLabs)
• AI purpose: Voice Generation; Text Style Replication
• Behaviour type: Tool
• Alleged developer: unknown
• Alleged deployer: unknown
• Alleged harmed parties: Perkins family, Ben Perkin's parents

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Minor, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Minor
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: The report describes two occurrences causing a total reported financial loss of 15,449 USD, with an average loss of 7,724.50 USD per occurrence.

Directly caused: Perkin's parents lost 15,449 USD (21,000 CAD) directly to the scammers via a Bitcoin transfer.

Inferred additional harm: Given the FTC statistics of 11 million USD lost to phone imposter scams in 2022, we infer substantial unquantified financial losses globally from similar AI voice scams.

Malicious content

Reported: The report explicitly describes the creation of synthetic cloned voices used for malicious impersonation.

Directly caused: The AI system was used to generate highly convincing synthetic voice clones of relatives to facilitate fraud.

Inferred additional harm: We infer that thousands of synthetic voice clones are being generated and spread across telecommunication networks to facilitate scams.

Privacy

Reported: The report explicitly describes the non-consensual harvesting of public voice recordings to train voice-cloning models.

Indirectly caused: Scammers harvested public audio recordings from social media platforms (YouTube, TikTok, Facebook) to train synthetic voice models without consent.

Inferred additional harm: Widespread unauthorized harvesting of personal audio data from social media platforms to train synthetic voice models.

Psychological

Reported: The report explicitly describes psychological distress, fear, and panic experienced by the elderly victims during the scam calls.

Indirectly caused: Victims experienced intense fear, panic, and subsequent embarrassment upon realizing they had been duped by a cloned voice of their loved one.

Inferred additional harm: Widespread anxiety and loss of trust in voice communications among elderly individuals targeted by similar voice-cloning scams.

Epistemic

Reported: The report explicitly describes epistemic harm through the generation of highly realistic fake audio that misled victims.

Directly caused: The AI generated highly realistic fake audio that successfully misled victims into believing their relatives were in danger.

Inferred additional harm: The erosion of trust in voice communications, as people can no longer reliably verify the identity of callers based on voice alone.

People affected

• Occurrences reported: 2
• People reportedly harmed: 4
• People reportedly exposed: 4

Potential causes

Management

• Delayed Safety Safeguards: Companies launched tools before implementing robust abuse safeguards.
• Priority on Rapid Release: Releasing beta platforms publicly before establishing abuse prevention.

Technology

• Highly Effective Voice Cloning: AI can clone a voice with highly realistic pitch, timbre, and accent.
• Low-Cost AI Voice Tools: Voice synthesis tools are cheap and highly accessible to anyone.
• Rapid Voice Analysis: AI only needs 3 to 30 seconds of audio to create a convincing clone.

Data Inputs

• Publicly Available Audio: Scammers harvest voice samples from public social media like TikTok.
• Small Sample Requirements: AI models require very short audio clips to clone voices.

Human Factors

• Emotional Manipulation: Scammers exploit fear and urgency, causing victims to react without thinking.
• Vulnerability of the Elderly: Elderly targets are more susceptible to high-pressure impersonation scams.
• Trust in Voice Authenticity: Victims naturally trust the familiar voice of their loved ones on the phone.

Process and Methods

• Anonymous Free Tiers: Free, anonymous accounts allowed bad actors to misuse voice cloning tools.
• Difficult Scammer Tracking: Global operations and spoofed numbers make tracing scammers nearly impossible.
• Untraceable Payment Methods: Use of Bitcoin and cash transfers prevents recovery of stolen funds.

Regulatory Environment

• Lack of AI Liability Laws: Courts have no clear precedent to hold AI developers liable for tool misuse.
• Insufficient Regulatory Control: Government regulators struggle to control the rapid spread of AI voice tools.
• Difficult Jurisdiction: Scammers operate globally, making law enforcement and jurisdiction complex.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts of the two primary incidents in Canada, including specific financial figures, names of victims, and the specific AI technology (ElevenLabs) used. The role of the AI system is explicitly stated, and there are no major conflicting details between the sources.
• Ambiguities identified: The exact mechanism by which the scammers obtained the specific voice samples of the victims' relatives is suspected (e.g., YouTube, Facebook) but not definitively proven.
• Alternative interpretations: None. The events are clearly identified as AI-assisted financial fraud.

This incident involved human scammers using ElevenLabs AI voice-cloning technology to target Canadian families in financial imposter scams. While causing individual financial and psychological distress, the national security impact remains minor and is manageable under standard law enforcement and consumer protection frameworks.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: Canada
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding the proliferation of consumer-grade generative AI tools for fraud.
• Autonomy: Human-controlled. The AI voice-cloning system was used as a tool directly controlled by human scammers to generate specific audio clips.
• Novelty: Evolved capability. Represents a significant advancement of existing imposter scams by integrating highly convincing AI-generated voice cloning.

Impact by dimension

• Physical security: Negligible. No physical threat, kinetic attacks, or critical infrastructure compromise occurred during these financial fraud incidents.
• Information security: Negligible. The synthetic media was used for localized financial fraud rather than state-sponsored disinformation or intelligence operations.
• Sovereignty: Negligible. The incident targeted private individuals and did not impact government operations, electoral systems, or state sovereignty.
• Economic security: Minor. Localized financial fraud resulting in individual losses, manageable through standard banking security and law enforcement procedures.
• Societal stability: Minor. Minor impact on societal trust in voice communications and privacy due to unauthorized harvesting of personal audio data.