AI-Generated Faces Used by Scammers to Pose as a Law Firm in Boston

GAN faces were allegedly used by scammers alongside a parked domain and a fake website to impersonate a Boston law firm.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The perpetrators used GAN-generated synthetic faces to construct a fake law firm website and impersonate legal professionals to extort backlinks from website owners for SEO manipulation.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The scam and extortion campaign were intentionally planned and executed by human actors who used GAN-generated faces as a tool to establish a fraudulent online identity.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The AI system was used to generate deepfake images (synthetic faces of non-existent people). Under the EU AI Act, such systems are subject to transparency obligations to ensure users are aware the content is AI-generated.

AI system and alleged parties

• AI system: GAN
• AI purpose: Deepfake Image Generation; Image Generation
• Behaviour type: Tool
• Alleged developer: unknown
• Alleged deployer: Unknown scammers
• Alleged harmed parties: email users

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 1

Potential causes

Management

• Lack of AI Ethics in SEO: Scammers exploit AI capabilities to optimize fraudulent SEO campaigns.

Technology

• Realistic GAN Face Generation: Generative AI creates highly realistic fake human faces.
• Presence of Visual Artifacts: GAN faces contain subtle anomalies like distorted earrings.

Data Inputs

• Large Facial Image Datasets: GANs require massive datasets of real faces to train effectively.

Human Factors

• Inability to Detect GAN Faces: Users fail to recognize synthetic images without technical training.
• Susceptibility to Social Engineering: People trust realistic-looking faces on professional websites.

Process and Methods

• Abuse of Free GAN Tools: Scammers easily access free websites that generate synthetic faces.

Regulatory Environment

• Lack of Synthetic Media Regulation: No legal frameworks prevent or penalize using AI faces for fraud.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides a clear, first-hand investigative account of the scam, detailing how the GAN-generated faces were identified and how the fraudulent operation was verified and shut down.

An online scam operation used GAN-generated synthetic faces to construct a fake law firm website and extort bloggers for SEO backlinks via fraudulent DMCA notices. The incident represents a minor national security concern, demonstrating an evolved capability of using synthetic media for social engineering, but was easily managed and lacked any state-actor involvement or critical infrastructure impact.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The specific campaign was dismantled, but the use of synthetic media for fraud represents an ongoing strategic concern.
• Autonomy: Human-controlled. The AI system was used as a static tool to generate images, while human operators designed the website and sent the extortion emails.
• Novelty: Evolved capability. Represents an evolved use of GAN-generated imagery to construct complete, fake professional organizations for social engineering and extortion.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or human safety were threatened by this digital scam.
• Information security: Minor. Perpetrators used synthetic GAN-generated faces to construct fake identities for a fraudulent law firm website to conduct social engineering, representing a minor, easily managed threat with no state-actor involvement.
• Sovereignty: Negligible. No impact on state authority, government operations, or constitutional processes.
• Economic security: Negligible. No strategic industries, financial systems, or critical technologies were targeted or compromised.
• Societal stability: Negligible. The incident was a localized fraud attempt targeting website owners and did not threaten social cohesion or civil liberties.