Multiple incidents have been reported across the United States where students used AI-powered 'nudification' applications to create and distribute nonconsensual sexually explicit images of their female classmates. These incidents have prompted state and federal lawmakers to rush new legislation to criminalize the creation and sharing of such deepfakes, as existing laws often fail to cover AI-generated content. The reports highlight a growing trend of technology-facilitated sexual harassment in schools, impacting the mental health and safety of minor victims.
At a high school in Issaquah, Washington, a male student is reported to have used deepfake technology to alter pictures of several female classmates and then shared them.
Risk classification
- Primary risk domain: 4 Malicious actors
- Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation
The incident involves the targeted creation and distribution of humiliating and sexually explicit synthetic imagery of specific victims (female classmates) for personal harassment and exploitation.
Additional risk subdomains
- 1.2 Exposure to toxic content: The creation and circulation of sexually explicit deepfakes of minors exposed students and school communities to highly toxic and inappropriate content.
Causal factors
- Entity: Human
- Intent: Intentional
- Timing: Post-deployment
The harm was caused by the intentional actions of male students who chose to use AI nudification apps to create and distribute sexually explicit deepfakes of their female classmates.
EU AI Act risk tier
- Risk tier: 3 Limited Risk
Risk Level 3: Limited Risk. The system is used to generate 'AI-generated content such as deepfakes', which falls under the transparency obligations of Limited Risk systems.
AI system and alleged parties
- AI system: AI nudification apps
- AI purpose: Deepfake Image Generation; Image Retouching
- Behaviour type: Tool
- Alleged developer: unknown
- Alleged deployer: Unnamed male student
- Alleged harmed parties: Anonymous female high school students
Harm severity
Severity ratings are based on this scale.
Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.
- Physical: direct Negligible, indirect Negligible
- Infrastructure: direct Negligible, indirect Negligible
- Property: direct Negligible, indirect Negligible
- Financial: direct Negligible, indirect Negligible
- Environmental: direct Negligible, indirect Negligible
- Malicious content: direct Substantial, indirect Negligible
- Differential treatment: direct Minor, indirect Negligible
- Civil rights: direct Minor, indirect Negligible
- Democracy: direct Negligible, indirect Negligible
- Privacy: direct Minor, indirect Negligible
- Psychological: direct Minor, indirect Negligible
- Epistemic: direct Minor, indirect Negligible
- Child sexual exploitation and abuse: direct Substantial, indirect Negligible
Malicious content
Reported: The report explicitly describes toxic and malicious content created and spread directly by the incident.
Directly caused: Male students used AI apps to generate and circulate fake nude images of female classmates via group chats on apps like Snapchat and Instagram.
Inferred additional harm: Potential further dissemination of these images online, where they could circulate indefinitely.
Differential treatment
Reported: The report explicitly describes differential treatment, as the harassment specifically targeted female students.
Directly caused: Male classmates targeted female classmates to create and distribute sexually explicit deepfakes, representing a gender-based form of harassment.
Civil rights
Reported: The report explicitly describes violations of human and civil rights caused directly by the incident.
Directly caused: The nonconsensual creation and distribution of sexually explicit depictions of minors violates their personal dignity, privacy, and civil rights.
Privacy
Reported: The report explicitly describes privacy violations caused directly by the incident.
Directly caused: Clothed photos of real girls were surreptitiously taken or used to generate simulated nude images without their consent.
Psychological
Reported: The report explicitly describes psychological and mental-health harm to the victims.
Directly caused: Victims felt 'extremely uncomfortable' and the reports note that these apps threaten girls' mental health and reputations.
Inferred additional harm: Widespread anxiety, distress, and trauma among the targeted girls at Issaquah High School and other schools mentioned.
Epistemic
Reported: The report explicitly describes epistemic harm through the creation of highly realistic fake images.
Directly caused: AI was used to fabricate realistic nude images of clothed girls, creating false representations of real people.
Child sexual exploitation and abuse
Reported: The report explicitly describes CSEA incidents involving AI-generated CSAM cases.
Directly caused: The creation of sexually explicit deepfakes of minors (e.g., 12 and 13-year-old girls in Miami-Dade, high school girls) was reported.
People affected
- Occurrences reported: 5
- People reportedly harmed: 20
- People reportedly exposed: 200
Potential causes
Management
- Commercialization of Nudification Apps: App developers profit from releasing tools that enable nonconsensual imagery.
Technology
- Availability of Nudification Apps: Apps designed to strip clothed photos of real people are widely accessible.
- Generative AI Image Manipulation: AI algorithms can realistically simulate nude images using real faces.
Data Inputs
- Use of Unconsented Clothed Photos: Target photos taken in public or social settings were used without consent.
Human Factors
- Peer Exploitation and Harassment: Male classmates surreptitiously generated and shared explicit fake images.
- Lack of Digital Responsibility: Minors lack understanding of the severe consequences of using generative AI.
Process and Methods
- Lack of App Developer Liability: No mechanisms exist to hold app developers accountable for misuse.
- Delayed School Notification: Schools failed to immediately inform parents of the deepfake victimization.
Regulatory Environment
- Inadequate Legal Protections: Existing laws failed to address AI-generated synthetic explicit images.
- Lack of Federal Legislation: No federal laws specifically criminalize the creation of deepfake nudes.
Information quality
- Classification confidence: High
- Reason for confidence: The reports provide clear, detailed accounts of multiple incidents of AI-generated deepfake creation in schools, including specific locations, legal responses, and the impact on victims. The role of the AI nudification apps is explicitly described.
- Ambiguities identified: The exact names of the AI apps used are not specified, and the precise number of victims at some of the schools is described generally rather than with exact figures.
- Alternative interpretations: None. The events clearly represent nonconsensual deepfake creation and harassment.
Multiple incidents of students using AI-powered nudification apps to create nonconsensual explicit deepfakes of classmates highlight growing societal challenges around synthetic media. While causing severe distress to victims and prompting legislative updates, the national security impact remains minor and localized to civil law enforcement domains.
- Overall national security impact: Minor
- Scope: Single nation
- Primary target: United States
- Alleged perpetrator: Students
Threat characteristics
- Imminence: Long-term. Represents an ongoing societal and regulatory challenge regarding synthetic media rather than an active national security crisis.
- Autonomy: Human-controlled. The AI nudification applications function as tools directly controlled and initiated by human operators to generate specific outputs.
- Novelty: Evolved capability. Represents a significant technological evolution of nonconsensual explicit imagery creation, lowering the barrier to entry through generative AI.
Impact by dimension
- Physical security: Negligible. No physical systems, critical infrastructure, or kinetic targeting capabilities were affected in this incident.
- Information security: Negligible. No state-sponsored information operations, intelligence compromise, or systematic propaganda campaigns were identified.
- Sovereignty: Negligible. Core government operations and sovereign authority remained intact, though state and federal lawmakers are proposing legislative updates.
- Economic security: Negligible. No strategic industries, financial systems, or AI technological competitive advantages were compromised.
- Societal stability: Minor. Represents a notable escalation in technology-facilitated sexual harassment targeting minor students, causing psychological distress and prompting legislative action, but managed via local law enforcement and schools.