Students of Richland School District in Cambria County, Pennsylvania Allegedly Used AI to Generate Obscene Images of Other Students

The Richland School District (Cambria County, Pennsylvania) released a statement on November 19, 2024 announcing an investigation into "some secondary students" who are alleged to have used AI to "create and electronically distribute obscene images of Richland students." The school administrators became aware of the incident on November 14, 2024.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

Students used AI to generate and distribute obscene, humiliating sexual imagery of specific peers, which fits the targeted creation of humiliating or sexual imagery under 4.3.

Additional risk subdomains

• 1.2 Exposure to toxic content: The creation and distribution of obscene images exposed students to harmful and inappropriate content.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was caused by students intentionally using a deployed AI system to generate and distribute obscene images of their peers.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The incident involves the creation of AI-generated content (deepfakes/obscene synthetic images), which falls under Risk Level 3 due to transparency and manipulation concerns.

AI system and alleged parties

• AI system: None named
• AI purpose: Deepfake Image Generation; Image Generation
• Behaviour type: Tool
• Alleged developer: Unknown deepfake technology developers
• Alleged deployer: Richland School District students
• Alleged harmed parties: Richland School District students

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Substantial, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Substantial, indirect Negligible

Malicious content

Reported: The report explicitly describes the creation and distribution of obscene and inappropriate images of students using AI.

Directly caused: Students used AI to directly generate obscene and inappropriate images of other students.

Child sexual exploitation and abuse

Reported: The report describes an incident where students used AI to create obscene images of other secondary students, prompting notifications to Children and Youth Services and Childline.

Directly caused: Students used AI to generate obscene images of secondary school students, who are minors, constituting AI-generated CSAM.

People affected

• Occurrences reported: 1
• People reportedly harmed: 2
• People reportedly exposed: 2

Potential causes

Technology

• Lack of AI Generation Safeguards: AI systems lacked filters to prevent generating obscene images of minors.
• Digital Distribution Platforms: Electronic channels facilitated the rapid spread of generated images.

Data Inputs

• Unauthorized Use of Target Photos: AI tools accepted photos of students without consent to generate deepfakes.

Human Factors

• Student Misuse of AI Tools: Students exercised poor judgment by using AI to harm classmates.

Information quality

• Classification confidence: Medium
• Reason for confidence: The report clearly outlines the event, the actors (students), and the immediate administrative and legal response. However, it lacks technical details about the specific AI tool used, how it was accessed, and the exact number of victims, requiring some inference.
• Ambiguities identified: The specific AI application used is not named, and the exact number of affected students is not specified.
• Alternative interpretations: The incident could be classified primarily under 1.2 (Exposure to toxic content), but boundary rule B1 directs targeted humiliation to 4.3.
• Missing information: The specific AI software/platform used, the number of victims, and the exact nature of the images.

Secondary students in Pennsylvania used generative AI to create and distribute non-consensual obscene images of peers. While causing localized harm and prompting law enforcement and child protective services involvement, the incident carries negligible direct implications for national security.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Secondary students

Threat characteristics

• Imminence: Long-term. The immediate local event is contained, representing a long-term societal challenge regarding the accessibility of consumer AI tools rather than an active national security crisis.
• Autonomy: Human-controlled. The AI system was used as a tool directly controlled and prompted by human actors to generate specific content.
• Novelty: Established threat. The use of generative AI by students to create non-consensual obscene imagery of peers is an established and increasingly common threat pattern.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or kinetic targeting capabilities were involved or compromised in this local school incident.
• Information security: Negligible. While involving synthetic media, the incident was a localized peer-to-peer event with no connection to state-sponsored information warfare or intelligence compromise.
• Sovereignty: Negligible. Core government operations, electoral systems, and national sovereignty remained entirely unaffected by this local school disciplinary and law enforcement matter.
• Economic security: Negligible. No strategic technologies, financial systems, or critical supply chains were targeted or compromised during this incident.
• Societal stability: Minor. The incident represents a localized violation of student privacy and dignity through AI-generated obscene imagery, which is being managed by local law enforcement and school authorities.