Quantum AI Scam Reportedly Used AI-Generated Celebrity Endorsements and Spoofed Media Sites to Solicit Investments

A global investment fraud network known as Quantum AI has reportedly used AI-generated deepfake videos, fake news sites, phishing pages, and spoofed calls to deceive victims. The scheme allegedly impersonates celebrities, news outlets, and political leaders to promote fraudulent trading platforms, leading targets to transfer money they cannot recover. Investigations suggest coordinated operations across multiple countries and platforms.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves malicious actors using AI-generated deepfakes of public figures to orchestrate highly targeted financial fraud and investment scams.

Additional risk subdomains

• 6.3 Economic and cultural devaluation of human effort: The unauthorized replication and exploitation of public figures' and journalists' likenesses devalues their professional reputation and human effort.
• 3.1 False or misleading information: The scam relies on AI-generated fake news broadcasts and articles that spread false information to deceive investors.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident is driven by human malicious actors intentionally deploying deepfake AI models post-deployment to defraud the public.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the use of AI-generated content such as deepfakes, which falls under Risk Level 3 due to transparency obligations requiring users to be informed they are interacting with AI-generated media.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: Unknown voice cloning technology developers, Unknown deepfake technology developers, Generative AI companies
• Alleged deployer: Quantum AI scammers, Quantum AI
• Alleged harmed parties: YouTube users, Victims of impersonation scams, Victims of deepfake-enabled fraud, Steph McGovern, Spencer & Stanley, social media users, Smithville International, Rishi Sunak, Richard Branson, Retail investors, Meta users, Martin Lewis, Mark Carney, Keir Starmer, Investors, General public, Forbes, Financial services customers, Elon Musk, Elderly investors, El Universo, DW News, Domenica In, David Beckham, Daily Mirror, Cryptocurrency investors, Consumers, CNN, BBC

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Substantial
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Minor
• Privacy: direct Negligible, indirect Substantial
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: Yes, the reports describe massive financial losses, including over 8 million USD lost in Australia alone, and individual losses ranging from 250 GBP to over 80,000 USD.

Indirectly caused: Victims lost millions of dollars globally, with an average loss of 765 GBP per victim in the UK and individual losses up to 80,000 USD in Australia.

Inferred additional harm: Given the global scale and hundreds of active domains, total financial losses are likely in the tens of millions of dollars.

Malicious content

Reported: Yes, the reports describe the creation and spread of malicious deepfake videos and fake news articles designed to defraud the public.

Directly caused: The incident directly involved the creation of over 100 deepfake video ads on Facebook and dozens of fake news articles across multiple platforms.

Inferred additional harm: It is likely that thousands of additional deepfake videos and malicious ads were generated and distributed globally across various social media platforms.

Democracy

Reported: Yes, the reports raise alarms about the risk AI deepfakes pose to democratic processes and elections by impersonating political leaders.

Indirectly caused: The widespread dissemination of deepfakes of political leaders like Rishi Sunak and Keir Starmer undermines trust in democratic institutions and political communication.

Inferred additional harm: The continued use of political deepfakes could lead to widespread voter manipulation and erosion of democratic integrity in upcoming global elections.

Privacy

Reported: Yes, the reports describe scammers harvesting personal information (names, phone numbers, emails) from victims who signed up on the fake websites.

Indirectly caused: Scammers obtained personal contact details of thousands of victims, which were then used to bombard them with fraudulent phone calls and emails.

Inferred additional harm: It is likely that the harvested personal data was sold or shared within cybercriminal networks, exposing victims to further privacy violations and future scams.

Psychological

Reported: Yes, the reports describe victims experiencing severe distress, anxiety, and ruin of lives due to losing savings.

Indirectly caused: Victims experienced severe emotional distress, anxiety, and trauma after losing their life savings to the scam.

Inferred additional harm: It is likely that thousands of victims suffered significant psychological distress and anxiety due to financial ruin, though specific numbers are not quantified.

Epistemic

Reported: Yes, the reports describe the widespread fabrication of news broadcasts, articles, and interviews, polluting the information ecosystem.

Directly caused: The campaign directly fabricated news reports from the BBC, CBC, and 9 News, creating false narratives about celebrity endorsements.

Inferred additional harm: The scale of these fabricated news articles contributes to a broader erosion of shared truth and public trust in legitimate news organizations.

People affected

• Occurrences reported: 1
• People reportedly harmed: 400
• People reportedly exposed: 1000000

Potential causes

Management

• Prioritization of Ad Revenue: Platforms accepted payments for unverified ads to boost revenue.
• Inadequate Proactive Risk Assessment: Management failed to deploy automated tools to detect deepfake ads.
• Lack of Platform Oversight: Failure to proactively audit verified accounts hijacked by scammers.

Technology

• Cheap Voice and Face Cloning: Allowed scammers to easily replicate public figures' likenesses.
• Lip-Syncing Software: Modified lip movements of figures to match AI-generated audio.
• Generative AI Tools: Enabled rapid creation of high-quality, convincing fake videos.

Data Inputs

• Authentic Interview Footage: Real media footage was hijacked and used as input for manipulation.
• Trusted Celebrity Likeness: Names and faces of trusted figures were fed into voice-cloning tools.
• Original Speech Audio: Voice recordings of public figures used to train cloning models.

Human Factors

• Public Trust in Celebrities: Victims trusted fake endorsements by respected public figures.
• Low Digital Literacy: Victims struggled to spot unnatural movements and lip-sync errors.
• Overreliance on Social Media: Users assumed platforms verified the legitimacy of paid advertisements.

Process and Methods

• Lax Ad Moderation Policies: Social media platforms failed to catch policy-violating deepfake ads.
• Delayed Content Takedowns: Slow response times allowed scam ads to reach hundreds of thousands.
• IP Address Obfuscation: Use of CDNs and reverse proxies hid scammers' hosting infrastructure.

Regulatory Environment

• Lagging Online Safety Laws: New duties under the Online Safety Act were not yet fully in force.
• No Mandatory Ad Verification: Regulators did not require platforms to verify financial advertiser registry.
• Lack of Platform Accountability: Absence of strict legal penalties for hosting fraudulent AI content.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide extensive, consistent, and highly detailed accounts of the Quantum AI scam network across multiple countries, including specific financial losses, platforms used, and technical details of the hosting infrastructure.

The Quantum AI fraud network represents an evolved threat where malicious actors leverage generative AI and voice cloning to execute highly convincing global investment scams. While the incident primarily constitutes retail financial fraud with minor direct national security impact, the scale of public figure impersonation and information pollution highlights growing challenges for societal trust and digital ecosystem integrity.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: UK, Australia, Canada, Ireland, Hong Kong, India, Brazil
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing, persistent threat of AI-enabled fraud and information pollution rather than a sudden, acute national security crisis.
• Autonomy: Human-controlled. The generative AI models are utilized as tools directly controlled by human operators to create deepfake media and cloned voices.
• Novelty: Evolved capability. Represents a significant advancement of traditional financial scams by integrating highly convincing generative AI deepfakes and voice cloning at scale.

Impact by dimension

• Physical security: Negligible. No physical security threats, kinetic attacks, or critical infrastructure compromise reported in connection with this financial scam.
• Information security: Minor. Widespread use of high-quality deepfakes of political leaders and news outlets pollutes the information ecosystem, though primarily driven by financial fraud rather than state-sponsored information warfare.
• Sovereignty: Minor. Impersonation of political figures like Rishi Sunak raises concerns for democratic trust, but there is no evidence of direct compromise to government operations or election systems.
• Economic security: Minor. While causing millions in losses to individuals globally, the financial fraud does not threaten macro-level national economic stability or critical financial infrastructure.
• Societal stability: Minor. Large-scale deployment of synthetic media erodes public trust in shared media and public figures, causing widespread financial and psychological distress to thousands of citizens.