Students in Maltese Schools Reportedly Being Targeted by Deepfake Nudes

Beginning sometime in October 2024, students ranging in age from 12 to 20 in schools in Malta were targeted in deepfake nude incidents. The attackers have been superimposing the faces of some of the students over the bodies of naked girls or using "nudify" apps to "undress" the photos, which have reportedly been sent to the victims over WhatsApp in order to humiliate them or in some cases blackmail them.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves the targeted creation of humiliating sexual imagery of specific victims to blackmail and harass them, which aligns with sub-domain 4.3 under Boundary Rule B1.

Additional risk subdomains

• 1.2 Exposure to toxic content: The incident involves the creation and distribution of non-consensual sexual content depicting minors, exposing victims and peers to highly toxic material.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The harm was caused by human perpetrators intentionally using deployed AI deepfake apps to generate and distribute non-consensual sexual imagery to humiliate and blackmail victims.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. The report describes the use of deepfake-generating apps, which fall under the category of AI-generated content requiring specific transparency obligations under the EU AI Act.

AI system and alleged parties

• AI system: nudify apps
• AI purpose: Deepfake Image Generation; Image Generation
• Behaviour type: Tool
• Alleged developer: Unknown deepfake technology developers
• Alleged deployer: Unknown deepfake creators in Malta, Unknown deepfake creators
• Alleged harmed parties: Students in Malta, Minors in Malta, General public of Malta

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Minor, indirect Minor
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Substantial, indirect Negligible

Malicious content

Reported: The report explicitly describes the creation and distribution of toxic and malicious content.

Directly caused: AI-powered apps were used to generate realistic, non-consensual naked images of minors, which were then shared on messaging platforms like WhatsApp.

Inferred additional harm: There is a high risk that these generated images could be uploaded to online forums frequented by pedophiles, further spreading the toxic content.

Civil rights

Reported: The report explicitly describes violations of human and civil rights, particularly regarding the privacy and protection of children.

Directly caused: The non-consensual sexualization of minors violates their fundamental rights to privacy, personal dignity, and protection from sexual exploitation.

Inferred additional harm: The widespread availability of these apps likely leads to systemic violations of children's rights across digital spaces.

Privacy

Reported: The report explicitly describes privacy violations through the non-consensual use of personal photos.

Directly caused: The personal, clothed photos of at least 25 girls were taken without their consent and processed by AI to generate fake naked images, violating their bodily and informational privacy.

Inferred additional harm: It is highly likely that thousands of other individuals have had their public photos scraped and processed by similar 'nudify' apps without their consent.

Psychological

Reported: The report explicitly describes psychological harm, mentioning humiliation, trauma, and the distress of being questioned by authorities.

Directly caused: At least 25 young girls suffered severe humiliation, distress, and trauma due to the creation and distribution of non-consensual naked deepfakes of themselves.

Indirectly caused: Parents of the victims experienced significant distress and faced a difficult dilemma regarding whether to report the abuse to the police.

Inferred additional harm: It is highly likely that many victims will suffer long-term psychological impacts, including anxiety, depression, and social withdrawal, affecting dozens of unreported victims.

Child sexual exploitation and abuse

Reported: The report explicitly describes a CSEA incident involving the generation of child sexual abuse material (CSAM).

Directly caused: Perpetrators generated realistic fake naked images of girls under the age of 18 (some as young as 12), which constitutes AI-generated CSAM.

Inferred additional harm: The ease of access to these apps suggests a widespread, unquantified generation of synthetic CSAM that may be distributed globally.

People affected

• Occurrences reported: 5
• People reportedly harmed: 25
• People reportedly exposed: 25

Potential causes

Management

• Lack of App Safety Controls: App creators fail to implement controls preventing abuse of their AI.

Technology

• AI Image Reconstruction: AI apps reconstruct realistic naked bodies from clothed photos.
• Accessible Deepfake Tools: Easy access to powerful AI image manipulation tools on mobile.

Data Inputs

• Unrestricted Photo Access: Publicly available photos of victims are used as input for AI.

Human Factors

• Malicious Intent by Peers: Classmates use AI tools for revenge, blackmail, or humiliation.
• Lack of Digital Responsibility: Minors generate and share harmful content without understanding impact.

Process and Methods

• Easy Sharing on Chat Apps: Rapid dissemination of AI deepfakes via platforms like WhatsApp.
• Reluctance to Report: Parents avoid reporting to prevent further trauma to victims.

Regulatory Environment

• Gaps in Criminal Responsibility: Perpetrators are often under the legal age of criminal responsibility.
• Inadequate Legal Frameworks: Existing laws fail to address rapid rise of AI-generated abuse.
• Systemic Delays in Justice: Slow legal processes deter victims' families from seeking justice.

Information quality

• Classification confidence: High
• Reason for confidence: The reports from Malta and Spain provide clear, factual accounts of the incidents, including the number of victims, their ages, the technology used, and the social/legal impacts. There is little ambiguity about the events.
• Ambiguities identified: The specific names of the deepfake apps or their developers are not identified.
• Alternative interpretations: None.

Classmates and peers in Malta and Spain utilized AI-powered apps to generate and distribute non-consensual sexual imagery of minors for blackmail and harassment. While constituting severe violations of privacy and child protection rights, the national security impact remains minor and localized, manageable through standard legal, educational, and social welfare frameworks.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Malta, Spain
• Alleged perpetrator: Individuals

Threat characteristics

• Imminence: Long-term. Represents an ongoing societal and regulatory challenge regarding AI-enabled harassment rather than an immediate national security crisis.
• Autonomy: Human-controlled. The AI system functions strictly as a tool, requiring direct human input and initiation to generate the deepfake images.
• Novelty: Evolved capability. While deepfakes are an established technology, the proliferation of easily accessible nudifying apps targeting minors represents a significant evolution in the accessibility of harmful AI tools.

Impact by dimension

• Physical security: Negligible. No physical security threats, kinetic attacks, or critical infrastructure compromises were reported in connection with this incident.
• Information security: Negligible. The incident involves localized harassment and blackmail rather than state-sponsored information warfare, intelligence compromise, or strategic espionage.
• Sovereignty: Negligible. There is no threat to state sovereignty, border control, or core government operations, though it has prompted domestic legal debates.
• Economic security: Negligible. No significant economic infrastructure, strategic technology theft, or financial systems were targeted or compromised.
• Societal stability: Minor. The incident represents a serious violation of children's privacy and human rights through the generation of synthetic child sexual abuse material, but the impact is localized and manageable via standard legal and social welfare systems.