Generative AI Allegedly Used to Facilitate $255,000 Real Estate Fraud Scheme

A real estate scam is reported to have used AI-generated phishing emails to impersonate a title company lawyer, tricking homebuyer Raegan Bartlo into wiring $255,000 to a fraudulent account. The emails were alleged to be convincing, with no grammatical errors or tone issues. Bartlo recovered part of the funds but lost $112,000.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves a scammer using generative AI to draft a highly persuasive email impersonating a lawyer to defraud a specific victim of $112,000.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was caused by a human scammer who intentionally used a deployed generative AI system to draft a highly convincing phishing email to defraud a homebuyer.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the use of a generative AI chatbot to produce text content, which falls under the category of chatbots and AI-generated content requiring transparency.

AI system and alleged parties

• AI system: ChatGPT, generative AI programs (OpenAI)
• AI purpose: Writing Assistant; Text Style Replication
• Behaviour type: Assistant
• Alleged developer: Unknown generative AI tools creators
• Alleged deployer: Unknown scammers
• Alleged harmed parties: Real estate market, Raegan Bartlo, Financial institutions

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Minor
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: The report describes a net financial loss of $112,000 from an initial $255,000 wire transfer, across 2 occurrences (one successful, one unsuccessful), averaging $56,000 per occurrence.

Indirectly caused: The victim lost $112,000 (net after recovering more than half of the $255,000) due to the AI-facilitated phishing scam. This represents an average loss of $56,000 across the two reported email occurrences.

Psychological

Reported: The report explicitly describes psychological distress experienced by the victim.

Indirectly caused: The victim, Raegan Bartlo, experienced significant distress, stating 'It was terrifying. You feel violated. I thought we lost everything' after realizing she had been scammed.

People affected

• Occurrences reported: 2
• People reportedly harmed: 2
• People reportedly exposed: 2

Potential causes

Technology

• Generative AI Text Generation: AI models generate highly realistic and convincing business emails.
• Voice Cloning Technology: AI can clone voices in seconds to mimic trusted professionals.
• Lowered Barrier to Entry: Free AI tools allow low-skilled actors to execute complex scams.

Data Inputs

• Compromised Transaction Data: Scammers hack accounts to gain specific transaction details.
• Public Real Estate Databases: Public listings provide persuasive details to add to scam emails.

Human Factors

• Transaction Fatigue: Buyers are fatigued by the process and act quickly to close.
• Lack of AI Phishing Awareness: Victims do not expect AI-generated emails to lack typical errors.
• Trust in Professional Personas: Buyers easily trust emails appearing to come from their lawyers.

Process and Methods

• Inadequate Wire Verification: Lack of out-of-band verification before wiring large sums.
• Insecure Email Communication: Using standard email for sensitive financial instructions.

Regulatory Environment

• AI Use is Not Inherently Illegal: Legality of AI tools makes tracking and prevention difficult.
• Lack of AI-Specific Laws: Absence of regulations targeting AI-enabled financial fraud.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides clear, firsthand accounts from the victim, who has a cybersecurity background, along with corroborating statements from FBI officials and cybersecurity experts regarding the rise of AI-generated phishing scams. While it is noted that it is difficult to prove AI was used in this specific case, the characteristics of the email strongly align with generative AI capabilities.
• Ambiguities identified: It is not 100% proven that AI was used to write the specific email, as noted by the victim and the FBI, though it is highly suspected due to the lack of typical grammatical errors.
• Alternative interpretations: The phishing email could have been written by a highly proficient English-speaking human scammer without the aid of AI.

A consumer was defrauded of 112,000 dollars using highly convincing, AI-generated phishing emails impersonating a title company lawyer. While demonstrating how generative AI lowers the barrier for sophisticated social engineering, the incident remains a localized criminal matter with negligible direct implications for national security.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding the proliferation of AI-enabled social engineering rather than an active cyber crisis.
• Autonomy: Human-controlled. The AI system acted as a writing assistant, while the human scammer retained full control over targeting and execution.
• Novelty: Evolved capability. Represents an evolved capability where generative AI is used to lower technical barriers and improve the quality of traditional phishing attacks.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or human safety threats were impacted by this localized financial fraud.
• Information security: Negligible. The incident represents localized financial fraud rather than systematic information warfare, intelligence compromise, or state-sponsored disinformation.
• Sovereignty: Negligible. No core government operations, electoral systems, or state sovereignty functions were disrupted or targeted.
• Economic security: Minor. Represents minor economic impact to individuals with no systemic threat to financial institutions, critical supply chains, or national strategic advantages.
• Societal stability: Negligible. No threats to societal stability, civil liberties, or population-scale social cohesion were observed.