Scammers Allegedly Use Deepfake Technology to Pose as Leonor, Princess of Asturias, in Fraud Scheme

Scammers have allegedly been using deepfake technology and fake social media accounts to reportedly impersonate Leonor, Princess of Asturias, targeting vulnerable individuals in Latin America. Victims were reportedly lured with promises of financial aid, requiring payments for "fees" or "taxes" before receiving funds, only to allegedly be defrauded repeatedly. TikTok accounts with thousands of followers reportedly amplified the scheme using AI to increase the appearance of credibility.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves scammers using AI-generated deepfakes to impersonate a royal figure and defraud specific vulnerable victims for financial gain.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The financial scam was intentionally orchestrated by human actors using deployed AI generation tools to create deepfakes.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk. The report describes the use of 'AI tools to mimic Leonor's voice and likeness' and 'a fake Princess Leonor — created using AI tools to mimic her movement and speech', which constitutes 'AI-generated content such as deepfakes'.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: Unknown deepfake technology developers
• Alleged deployer: Unknown scammers, TikTok scammers, Scammers posing as Leonor (Princess of Asturias)
• Alleged harmed parties: Vulnerable people in Latin American countries, Juana Cobo, Financially distressed individuals

Harm severity

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Minor
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Minor, indirect Minor
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes malicious content created and spread directly by the incident.

Directly caused: Scammers created AI-generated deepfake videos of Princess Leonor of Spain to deceive users into participating in fake contests and financial aid schemes.

Indirectly caused: TikTok's algorithm amplified these videos, allowing them to reach millions of views and gain hundreds of thousands of followers.

Inferred additional harm: It is likely that numerous other AI-generated deepfake videos and audio clips were created and distributed across various social media platforms to target vulnerable individuals.

Differential treatment

Reported: The report explicitly describes differential treatment of individuals caused directly by the incident.

Directly caused: The scammers intentionally targeted vulnerable, low-income, and elderly individuals (specifically those over 60) in Latin America for exploitation.

Indirectly caused: N/A

Inferred additional harm: The systemic targeting of vulnerable populations in developing countries exploits existing socioeconomic inequalities, further disadvantaging these groups.

Privacy

Reported: The report explicitly describes privacy violations caused directly by the incident.

Directly caused: Scammers solicited personal information, including phone numbers and bank account details, from victims via TikTok comments and private messages.

Indirectly caused: N/A

Inferred additional harm: It is likely that the personal data collected from thousands of users is being stored, shared, or sold for further malicious activities.

Psychological

Reported: The report explicitly describes psychological distress and false hope experienced by victims of the scam.

Directly caused: Victims like Cobo experienced emotional manipulation, getting their hopes up before realizing they were defrauded.

Indirectly caused: The financial debt incurred by victims caused ongoing anxiety and distress.

Inferred additional harm: It is likely that many other vulnerable and elderly victims experienced similar emotional distress, anxiety, and shame from being defrauded, though specific details are not provided.

Epistemic

Reported: The report explicitly describes epistemic harm caused directly by the incident.

Directly caused: AI tools were used to generate realistic videos and speech of Princess Leonor, fabricating her endorsement of a financial aid program.

Indirectly caused: The proliferation of these deepfakes on TikTok misled hundreds of thousands of followers into believing the Spanish Royal Family was offering financial aid.

Inferred additional harm: The widespread use of royal deepfakes erodes public trust in official communications and the authenticity of digital media.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 410000

Potential causes

Management

• Failure to Proactively Monitor: TikTok relied on external reports rather than proactively detecting royal fakes.

Technology

• AI Voice and Likeness Mimicry: AI tools used to mimic Princess Leonor's voice and movement to deceive victims.
• Engagement-Driven Algorithms: TikTok's algorithm amplified fake accounts due to high user engagement.

Data Inputs

• Lack of Official Royal Accounts: Absence of real royal profiles allowed scammers to fill the identity vacuum.
• Manipulated Profile Images: Scammers added fake blue verification checks to profile pictures to gain trust.

Human Factors

• Trust in Royal Authority: Victims trusted the royal identity, making them susceptible to the scam.
• Vulnerability of Target Audience: Elderly and financially vulnerable individuals were targeted for exploitation.

Process and Methods

• Slow Content Moderation: TikTok was slow to remove reported fake accounts, letting some stay active.
• Inadequate Account Verification: Lack of robust verification allowed fake accounts to mimic verified profiles.

Regulatory Environment

• Lack of Cross-Border Enforcement: Scammers operated from Dominican Republic, hindering local law enforcement.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts of the scam, including a specific victim interview, the platforms used, the role of AI deepfakes, and the response from authorities and TikTok. There is high certainty regarding the classification of the incident as a financial scam utilizing AI deepfakes.
• Ambiguities identified: The specific AI tools or models used by the scammers to generate the deepfakes are not identified.
• Alternative interpretations: None. The incident is clearly a coordinated financial fraud campaign using impersonation and deepfakes.

An international financial fraud scheme utilized AI-generated deepfakes of Princess Leonor of Spain to target vulnerable individuals in Latin America. While the incident demonstrates the evolving threat of synthetic media in facilitating cross-border crime, its national security impact remains minor as it lacks geopolitical motivation or systemic infrastructure targets.

• Overall national security impact: Minor
• Response level: Moderate
• Scope: Multiple nations
• Primary target: Latin America
• Other affected: Spain, Dominican Republic
• Alleged perpetrator: Scammers in the Dominican Republic

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding the proliferation of deepfake financial scams rather than an immediate national security crisis.
• Autonomy: Human-controlled. The AI systems were used as tools by human scammers who directed the content generation and manually executed the fraud scheme.
• Novelty: Evolved capability. Represents an evolved capability where deepfake technology is combined with social media algorithms to target specific international demographics.

Impact by dimension

• Physical security: Negligible. No physical threat, kinetic attacks, or critical infrastructure manipulation occurred in this incident.
• Information security: Minor. AI deepfakes impersonated a member of the Spanish Royal Family, but the operation was a financially motivated scam rather than state-sponsored geopolitical disinformation.
• Sovereignty: Negligible. No core government operations, electoral systems, or sovereign decision-making processes were compromised.
• Economic security: Negligible. The incident involves localized financial fraud against individuals rather than systemic threats to national financial systems or critical supply chains.
• Societal stability: Minor. The scam systematically targeted vulnerable and elderly populations in Latin America, causing individual financial distress, but did not threaten large-scale social cohesion.