Scammers are increasingly using generative AI tools, such as voice cloning and AI-generated 'ghost sites', to execute highly convincing financial fraud and impersonation scams, resulting in a Georgia couple losing over $800,000.
A retired Gwinnett County, Georgia, couple reported losing nearly $800,000 after a stranger contacted Jerry Dunaway on WhatsApp and encouraged cryptocurrency trading. Police reportedly told the couple that scammers used an AI-generated "ghost site" to manipulate the account information he saw. The specific AI tool and developer were not identified. The loss was discovered when a larger withdrawal failed.
Risk classification
- Primary risk domain: 4 Malicious actors
- Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation
The incident involves scammers using AI voice cloning and AI-generated ghost sites to execute highly targeted financial fraud and impersonation scams against specific victims.
Additional risk subdomains
- 3.1 False or misleading information: The scammers utilized an AI-generated 'ghost site' to present fabricated financial data and false investment gains to deceive the victim.
Causal factors
- Entity: Human
- Intent: Intentional
- Timing: Post-deployment
The risk is caused by malicious human actors intentionally deploying AI tools to defraud victims after the models have been trained and released.
EU AI Act risk tier
- Risk tier: 3 Limited Risk
Limited Risk: The reports describe the use of generative AI to create deepfakes, such as cloned voices and fake photographs, which fall under the category of AI-generated content requiring transparency obligations.
AI system and alleged parties
- AI system: None named
- AI purpose: Voice Generation; Deepfake Image Generation
- Behaviour type: Tool
- Alleged developer: Generative AI developers
- Alleged deployer: Scammers, Investment scam operators, Cybercriminals, Cryptocurrency scammers
- Alleged harmed parties: Mindy Dunaway, Jerry Dunaway, Financial scam victims, Elderly investors, Elderly individuals, Cryptocurrency scam victims
Harm severity
Severity ratings are based on this scale.
Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.
- Physical: direct Negligible, indirect Negligible
- Infrastructure: direct Negligible, indirect Negligible
- Property: direct Negligible, indirect Negligible
- Financial: direct Minor, indirect Negligible
- Environmental: direct Negligible, indirect Negligible
- Malicious content: direct Minor, indirect Negligible
- Differential treatment: direct Minor, indirect Negligible
- Civil rights: direct Negligible, indirect Negligible
- Democracy: direct Negligible, indirect Negligible
- Privacy: direct Minor, indirect Negligible
- Psychological: direct Minor, indirect Minor
- Epistemic: direct Negligible, indirect Negligible
- Child sexual exploitation and abuse: direct Negligible, indirect Negligible
Financial
Reported: The reports describe massive financial losses. Jerry and Mindy Dunaway lost approximately $802,000. The FBI report notes $893 million in AI-enabled scam losses nationally, with $39.9 million in Florida and $10.4 million in Georgia. For the 2 specific occurrences, the average loss is approximately $401,000.
Directly caused: Jerry and Mindy Dunaway directly lost $802,000 due to the cryptocurrency scam.
Inferred additional harm: Based on the state-level data, hundreds of other victims in Florida and Georgia lost a combined total of over $50.3 million ($39.9M in FL and $10.4M in GA) due to similar AI-enabled scams.
Malicious content
Reported: The reports describe the creation of malicious content used to deceive victims.
Directly caused: Scammers created an AI-generated 'ghost site' to display fake screens and balances, and cloned the voice of an immigration attorney to target clients.
Inferred additional harm: Scammers likely generated thousands of cloned voice clips and fake emergency messages targeting hundreds of other victims across the US.
Differential treatment
Reported: The reports note that older adults are disproportionately targeted by these scams.
Directly caused: Jerry Dunaway (74) and his wife (73) were targeted, aligning with the report's note that most victims of these cryptocurrency investment scams are over the age of 60.
Inferred additional harm: Elderly individuals are systematically targeted by AI-enabled scams due to perceived vulnerabilities, affecting thousands of seniors nationally.
Privacy
Reported: The reports describe scammers researching family members and cloning voices.
Directly caused: Scammers harvested personal data and social media audio samples to clone the voice of the immigration attorney and research family relationships for grandparent scams.
Inferred additional harm: Thousands of individuals have had their social media audio and personal relationships scraped and analyzed by AI tools without consent to facilitate targeted scams.
Psychological
Reported: The report explicitly describes severe psychological distress and emotional harm experienced by the victims.
Directly caused: Jerry Dunaway experienced severe distress, feeling 'sick', 'embarrassed', and 'ashamed' after losing his life savings.
Indirectly caused: Mindy Dunaway experienced significant distress, comparing the emotional impact of the scam to her battle with metastasized cancer.
Inferred additional harm: It is highly likely that other scam victims (such as the cloned attorney and the hundreds of other reported victims in FL and GA) suffered similar severe emotional distress, anxiety, and shame, affecting an estimated 400+ people.
People affected
- Occurrences reported: 2
- People reportedly harmed: 3
- People reportedly exposed: 3
Potential causes
Management
- Inadequate Risk Mitigation: Financial and tech companies fail to implement robust AI fraud safeguards.
- Lack of User Education: Platforms do not proactively warn or educate users about AI scam tactics.
Technology
- AI Voice Cloning Tools: Scammers clone voices from social media to impersonate family members.
- AI-Generated Ghost Sites: AI-generated interfaces show fake balances and buttons to deceive victims.
- Generative AI Impersonation: Generative AI creates highly realistic fake photos and profiles to build trust.
Data Inputs
- Social Media Audio Exposure: Publicly available audio samples are harvested to train voice-cloning models.
- Personal Relationship Data: Scammers research family connections to make emergency requests seem authentic.
- Spoofed Contact Information: Legitimate phone numbers are spoofed to bypass caller verification.
Human Factors
- Cognitive Bias and Trust: Victims trust realistic cloned voices and professional-looking ghost sites.
- Emotional Manipulation: Scammers create artificial urgency, such as fake family emergencies.
- Lack of Technical Awareness: Victims are unaware of the sophistication of AI-driven scam capabilities.
Process and Methods
- Ineffective Identity Verification: Lack of verification methods like family code words to confirm identity.
- Weak App Authentication: Trading apps and communication platforms lack robust identity checks.
Regulatory Environment
- Lack of AI Tool Regulations: Absence of strict regulations on the accessibility of voice-cloning tools.
- Insufficient Platform Oversight: Weak regulatory enforcement on communication channels like WhatsApp.
Information quality
- Classification confidence: High
- Reason for confidence: The reports provide clear, detailed accounts of a specific cryptocurrency scam targeting the Dunaways, including the exact financial loss ($802,000) and the specific role of AI (generating a 'ghost site' and cloning voices). The FBI statistics provide solid context on the broader scale of AI-enabled scams, allowing for highly confident classifications.
- Ambiguities identified: The exact technical mechanism of the AI-generated 'ghost site' is not fully detailed, and it is unclear which specific AI tools were used to clone the attorney's voice.
- Alternative interpretations: The incident could be viewed purely as a traditional cybercrime/phishing scam where AI was merely incidental, rather than an AI-driven safety failure.
Widespread financial fraud and impersonation scams utilizing generative AI tools, such as voice cloning and 'ghost sites', have led to significant cumulative losses for US citizens. While representing an advanced cybercrime threat, the national security impact remains minor as it does not target critical infrastructure, state sovereignty, or systemic economic stability.
- Overall national security impact: Minor
- Scope: Single nation
- Primary target: United States
- Alleged perpetrator: Unknown
Threat characteristics
- Imminence: Long-term. Represents a persistent, ongoing cybercrime trend and strategic capability concern rather than an imminent national security crisis requiring immediate intervention.
- Autonomy: Human-controlled. AI tools like voice cloning and site generation are human-controlled and deployed manually by scammers to facilitate targeted social engineering.
- Novelty: Evolved capability. Represents an evolved capability where traditional phishing and social engineering methods are significantly enhanced by generative AI tools.
Impact by dimension
- Physical security: Negligible. No physical infrastructure, control systems, or human safety elements were targeted or compromised in these financial fraud incidents.
- Information security: Negligible. The incident involves localized financial scams and identity impersonation rather than coordinated state-sponsored information operations, mass surveillance, or intelligence compromise.
- Sovereignty: Negligible. No core government functions, sovereignty, border controls, or electoral processes were targeted or disrupted.
- Economic security: Minor. While cumulative national losses from AI-enabled fraud are high ($893 million), the threat consists of distributed consumer fraud rather than systemic threats to critical financial infrastructure or strategic technology.
- Societal stability: Minor. Targeted scams cause significant individual distress, financial harm, and disproportionately target elderly populations, but do not threaten overall societal stability, civil liberties, or public trust at a national scale.