.avif){kind=small}
The report details the widespread use of AI to create non-consensual pornographic deepfakes, specifically targeting women. The victim, Breeze [PERSON_006], experienced severe psychological trauma after her image was used in over 800 deepfake links. The article highlights the role of search engines and platforms in monetizing this content and the lack of clear legal recourse due to Section 230 protections.
Porn sites are alleged to have used AI-generated images of Breeze Liu without her consent, leading to severe emotional distress. Liu discovered a video of herself on Pornhub, which was then deepfaked and spread across over 800 links. Despite efforts to remove the content, many sites refused to comply, perpetuating the violation and exploitation of her image.
The primary incident involves the creation of humiliating and sexual deepfake imagery targeting a specific victim for harassment and monetization, which fits the boundary rule for targeted exploitation.
The harm was caused by malicious human actors intentionally using deployed AI generation tools to create and distribute non-consensual deepfakes of victims.
Risk Level 3: Limited Risk. The report describes the generation of deepfakes, which are explicitly classified under Limited Risk, requiring transparency obligations to ensure users are aware the content is AI-generated.
Severity ratings are based on this scale.
Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.
Reported: Yes, the report describes the creation and spread of pornographic deepfakes.
Directly caused: Over 800 deepfake links of Breeze were circulated across malicious websites and Pornhub.
Inferred additional harm: It is inferred that millions of toxic, non-consensual pornographic deepfake videos of women and girls are spread across the internet.
Reported: Yes, the report highlights that 99 percent of the targets of pornographic deepfakes are women and girls.
Directly caused: Women and girls are disproportionately targeted and victimized by non-consensual deepfake pornography, leading to systemic gender-based harassment.
Inferred additional harm: This disproportionate targeting reinforces gender inequality and creates a hostile online environment for women and girls globally.
Reported: Yes, the report describes violations of personal dignity, bodily autonomy, and privacy.
Directly caused: Breeze's right to consent and personal dignity were violated by the unauthorized creation and distribution of sexually explicit deepfakes.
Inferred additional harm: The widespread generation of non-consensual sexual imagery violates the fundamental human rights to privacy and dignity of thousands of victims.
Reported: Yes, the report briefly mentions a deepfake robocall of Joe Biden targeting voters in New Hampshire and a fake video of Taylor Swift endorsing Donald Trump.
Directly caused: The robocall of Joe Biden aimed to influence voters during the New Hampshire primary, potentially undermining democratic processes.
Inferred additional harm: The proliferation of political deepfakes poses a significant threat to the integrity of elections and public trust in democratic institutions.
Reported: Yes, the report describes non-consensual image abuse and violation of privacy.
Directly caused: Breeze's likeness was used without her consent to generate and distribute explicit pornographic videos across the internet.
Inferred additional harm: Thousands of individuals have their privacy violated through the unauthorized use of their personal likenesses to generate explicit synthetic media.
Reported: Yes, the report explicitly describes severe psychological harm to the victim, Breeze [PERSON_006].
Directly caused: Breeze experienced devastating trauma, shame, suicidal ideation, and panic attacks, describing the experience as feeling like a part of her had permanently died.
Inferred additional harm: It is highly likely that thousands of other women and girls targeted by non-consensual deepfake pornography suffer similar severe psychological trauma and distress.
Reported: Yes, the report describes the creation of fake videos and robocalls that mislead the public.
Directly caused: Voters received a fake robocall mimicking Joe Biden, and a fake video of Taylor Swift endorsing Donald Trump was circulated, spreading false information.
Inferred additional harm: The widespread availability of deepfake technology erodes public trust in digital media and makes it difficult for individuals to distinguish between real and fabricated content.
The incident details the widespread use of generative AI to create non-consensual pornographic deepfakes and political disinformation, such as a fake Joe Biden robocall. While primarily a severe societal and civil rights issue, the technology also presents minor national security concerns regarding election integrity and information warfare.