The 'Nomani' phishing campaign, active throughout the second half of 2024, utilized AI-generated deepfakes and video testimonials to impersonate legitimate services and celebrities. These assets were used to lure victims into sophisticated investment scams, resulting in significant financial and data loss. The campaign leveraged social media malvertising and stolen accounts to distribute phishing links, demonstrating a coordinated effort by threat actors to exploit AI for financial fraud.
AI-generated deepfakes were reportedly used in the "HTML/Nomani" phishing campaign to mimic legitimate platforms like booking services and lured victims into investment scams. These scams allegedly leveraged realistic fake content to deceive users on social media for the purposes of financial fraud. This campaign was part of the rising misuse of AI in cybercrime during the second half of 2024.
Risk classification
- Primary risk domain: 4 Malicious actors
- Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation
The Nomani campaign used AI-generated deepfakes of celebrities to impersonate trusted individuals and execute fraudulent investment scams for financial gain.
Additional risk subdomains
- 2.1 Compromise of privacy by obtaining, leaking or correctly inferring sensitive information: The phishing infrastructure set up via the AI-driven lures harvested victims' sensitive personal information, including IDs and credit card details.
Causal factors
- Entity: Human
- Intent: Intentional
- Timing: Post-deployment
The incident was caused by human threat actors intentionally deploying AI-generated deepfakes post-deployment to execute financial scams.
EU AI Act risk tier
- Risk tier: 3 Limited Risk
Limited Risk: The system involves AI-generated content such as deepfakes. According to the definitions, Risk Level 3 applies to 'AI-generated content such as deepfakes' which require transparency obligations.
AI system and alleged parties
- AI system: unspecified
- AI purpose: Deepfake Video Generation; Voice Generation
- Behaviour type: Tool
- Alleged developer: Unknown deepfake technology developers
- Alleged deployer: Unknown scammers, HTML/Nomani
- Alleged harmed parties: Phishing victims, Booking.com customers, Booking.com, Airbnb users, Airbnb
Harm severity
Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.
- Physical: direct Negligible, indirect Negligible
- Infrastructure: direct Negligible, indirect Negligible
- Property: direct Negligible, indirect Negligible
- Financial: direct Substantial, indirect Negligible
- Environmental: direct Negligible, indirect Negligible
- Malicious content: direct Minor, indirect Negligible
- Differential treatment: direct Negligible, indirect Negligible
- Civil rights: direct Negligible, indirect Negligible
- Democracy: direct Negligible, indirect Negligible
- Privacy: direct Substantial, indirect Negligible
- Psychological: direct Negligible, indirect Negligible
- Epistemic: direct Minor, indirect Negligible
- Child sexual exploitation and abuse: direct Negligible, indirect Negligible
Financial
Reported: The report describes a related South Korean fraud network (MIDAS) that defrauded victims of nearly $6.3 million, though the specific Nomani campaign losses are not fully quantified but described as growing by 335%.
Directly caused: Average financial loss per occurrence is not explicitly detailed for the Nomani campaign, but the related MIDAS operation caused $6.3 million in losses.
Indirectly caused: N/A
Inferred additional harm: Given the 335% growth of the Nomani campaign and over 100 new URLs daily, total financial losses across thousands of victims are highly likely to be in the millions of dollars.
Malicious content
Reported: The report explicitly describes the creation and spread of AI-powered video testimonials featuring famous personalities used as fraudulent ads.
Directly caused: AI-generated deepfake video testimonials of famous personalities were spread via social media malvertising, Threads, Messenger, and Google reviews.
Indirectly caused: N/A
Inferred additional harm: Thousands of social media users were likely exposed to these malicious deepfake videos across multiple platforms.
Privacy
Reported: The report explicitly describes the harvesting of personal information, IDs, and credit card details from victims.
Directly caused: Phishing websites harvested victims' contact information, IDs, and credit card details.
Indirectly caused: N/A
Inferred additional harm: Given the scale of over 100 new URLs daily, thousands of victims likely had their sensitive personal and financial data compromised.
Epistemic
Reported: The report describes the use of AI-generated deepfake video testimonials of famous personalities to fabricate endorsements.
Directly caused: AI deepfakes fabricated video testimonials of famous personalities to deceive users into believing the investment schemes were legitimate.
Indirectly caused: N/A
Inferred additional harm: The widespread dissemination of deepfakes erodes public trust in video evidence and digital media authenticity.
People affected
- Occurrences reported: 1
- People reportedly harmed: 100
- People reportedly exposed: 10000
Potential causes
Management
- Weak Platform Moderation Oversight: Meta and Google fail to manage and police fraudulent profiles and reviews.
- Inadequate Scam Tracking: Delayed response in identifying and taking down malicious domains.
Technology
- AI-Powered Video Testimonials: Deepfake videos of famous figures are used to build trust with victims.
- Malicious Screen Capture Software: Illicit programs spy on users' screens to collect unauthorized data.
- Automated Phishing Infrastructure: Rapid generation of daily new URLs evades security detection systems.
Data Inputs
- Harvested Personal Information: Phishing forms collect victim contact details to enable direct phone scams.
- Stolen Profile Data: Abuse of legitimate accounts and profiles to distribute fraudulent ads.
- Real-Time Stock Market Data: Programs use real brokerage data to display fake trading charts.
Human Factors
- Susceptibility to Social Engineering: Victims are manipulated into trusting fake recovery lures and phone calls.
- Overconfidence in Ad Platforms: Users trust ads appearing on major social media platforms and Google.
- Trust in Impersonated Authorities: Targeting previously scammed victims using fake Europol and Interpol lures.
Process and Methods
- Inadequate Ad Verification: Social media platforms fail to detect and block AI-generated scam ads.
- Evading Bank Verification: Social engineering tactics bypass bank authorization and verification calls.
- Segmented Cybercrime Operations: Dividing tasks among specialized groups makes the scam chain highly efficient.
Regulatory Environment
- Lack of Global Ad Regulation: Weak cross-border enforcement allows foreign threat actors to run scam ads.
- Inadequate Identity Verification: Easy registration of fake profiles on social platforms enables rapid scaling.
Information quality
- Classification confidence: High
- Reason for confidence: The report clearly details the mechanics of the Nomani phishing campaign, the role of AI-generated deepfakes as lures, and the resulting financial and data harms. While exact victim counts for Nomani are not specified, the overall threat landscape and campaign growth are well-documented by ESET.
- Ambiguities identified: The exact number of victims and total financial losses specifically for the Nomani campaign are not separated from the South Korean MIDAS operation.
The Nomani campaign represents an evolved cybercrime threat leveraging AI-generated deepfakes for financial fraud and data harvesting. While causing millions in losses and exploiting thousands of victims globally, its national security impact remains minor as it does not target critical infrastructure, state sovereignty, or strategic economic systems.
- Overall national security impact: Minor
- Response level: Moderate
- Scope: Multiple nations
- Primary target: No clear primary
- Other affected: South Korea
- Alleged perpetrator: Suspected Russian-speaking threat actors
Threat characteristics
- Imminence: Long-term. Represents an ongoing strategic cybercrime trend and capability development rather than an imminent threat to national security.
- Autonomy: Human-controlled. AI was used as a tool to generate deepfake media assets, while humans managed the distribution, phishing infrastructure, and direct social engineering calls.
- Novelty: Evolved capability. Represents a significant evolution of traditional financial scams by integrating high-quality, automated AI-generated deepfake testimonials to increase credibility.
Impact by dimension
- Physical security: Negligible. No physical systems, critical infrastructure, or kinetic assets were targeted or affected by this financial scam campaign.
- Information security: Minor. Utilized AI-generated deepfakes of celebrities for social media malvertising, but the intent was financial fraud rather than state-sponsored geopolitical influence or intelligence compromise.
- Sovereignty: Negligible. The campaign did not target government decision-making, electoral systems, or core state authority.
- Economic security: Minor. Caused millions in financial losses to individual victims, including a related $6.3 million fraud network in South Korea, but does not threaten strategic national economic stability.
- Societal stability: Minor. Led to widespread personal data theft (IDs, credit cards) and emotional distress for defrauded individuals, but did not cause mass civil unrest or systematic rights violations.
.avif){kind=small}