.avif){kind=small}
Schools across Australia are experiencing a surge in the use of AI-powered 'nudify' apps to create and circulate non-consensual deepfake pornography of students and teachers. Perpetrators, often students themselves, use these tools to generate graphic images from social media photos, leading to severe psychological trauma for victims. The incident at Bacchus Marsh Grammar, where a student was arrested for creating images of 50 classmates, highlights the widespread nature of this abuse. Regulators and the government are now working to implement stricter laws and standards to hold platforms and users accountable for this form of image-based abuse.
Throughout 2024, schools in Australia dealt with a significant rise and proliferation of non-consensual deepfake pornography of students. Often, male students are reported to use "nudify" apps such as Undress AI with images of their classmates and teachers. Many of the sites have remained legal and accessible to minors, who in turn are using the sites to generate pornography of their peers.
The incident involves the targeted creation of humiliating and sexual deepfake imagery of specific victims for personal harassment and abuse.
The harm was driven by the intentional actions of human users utilizing a deployed AI tool to target and humiliate specific individuals.
Limited Risk: The system is used to generate AI-generated content such as deepfakes, which falls under Limited Risk and carries specific transparency obligations under the EU AI Act.
Severity ratings are based on this scale.
Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.
Reported: The report explicitly describes toxic and malicious content created and spread directly and indirectly by the incident.
Directly caused: The AI system was used to generate highly graphic, realistic fake pornographic images ('deepnudes') of around 50 female students and a teacher.
Indirectly caused: The generated explicit images were circulated to others via social media platforms like Instagram and Snapchat.
Inferred additional harm: Thousands of other deepnude images of children and adults are likely generated daily by the app's 100,000+ daily users.
Reported: The report explicitly describes differential treatment of individuals or groups caused directly by the incident.
Directly caused: The abuse overwhelmingly targeted female students and a female teacher, subjecting them to gender-based harassment and degradation.
Inferred additional harm: Widespread gender-based targeting in schools creates an unequal and hostile educational environment for female students.
Reported: The report explicitly describes violations of human and civil rights caused directly by the incident.
Directly caused: The incident violated the victims' fundamental rights to safety, personal dignity, and freedom from non-consensual sexual exploitation.
Inferred additional harm: The easy availability of these tools leads to systemic violations of children's rights to protection from sexual abuse and exploitation.
Reported: The report explicitly describes privacy violations caused directly by the incident.
Directly caused: The perpetrator harvested photos from the victims' social media accounts (some of which were private) to generate intimate, non-consensual deepfakes.
Inferred additional harm: The training of Undress AI relied on scraping millions of real images and data from across the internet without the subjects' consent.
Reported: The report explicitly describes severe psychological and mental-health harm to people caused directly and indirectly by the incident.
Directly caused: The report describes severe psychological trauma and distress among the 50 female victims at Bacchus Marsh Grammar and a teacher at Salesian College.
Indirectly caused: Parents and families experienced significant distress, anxiety, and worry regarding the ongoing safety of their children.
Inferred additional harm: Many of the 51 victims likely suffered long-term anxiety, depression, and social difficulties that were not fully detailed in the report.
Reported: The report explicitly describes epistemic harm caused directly by the incident.
Directly caused: The AI system fabricated highly realistic but entirely false visual evidence depicting the victims in explicit sexual scenarios.
Inferred additional harm: The proliferation of realistic deepfakes erodes public trust in visual media and personal photos.
Reported: The report explicitly describes a CSEA incident caused directly and indirectly by the technology.
Directly caused: The incident involved the generation of child sexual abuse material (CSAM) depicting around 50 schoolgirls who are minors.
Indirectly caused: The report notes that paedophiles are using such apps to create and share child sexual abuse material.
Inferred additional harm: Widespread, unquantified generation of CSAM by users exploiting the lack of age-verification or content controls on deepnude platforms.
The incident involves the widespread use of AI-powered 'nudify' apps by students to generate non-consensual explicit imagery of classmates and teachers in Australia. While causing severe psychological trauma and prompting legislative reforms, it remains a domestic law enforcement and regulatory issue rather than a threat to national security.