AI 'Nudify' Apps Used as Tools for Blackmail and Extortion

AI 'nudify' apps are being used to generate hyperrealistic non-consensual nude photos of individuals, which are then exploited for extortion and harassment. These apps use generative AI to remove clothing from images and create convincing fakes, often distributed on platforms like Telegram. Victims are threatened or shamed using these AI-generated images, with little recourse for removal or legal action.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves malicious actors using AI-generated non-consensual sexual imagery to blackmail and target specific victims for financial gain.

Additional risk subdomains

• 1.2 Exposure to toxic content: The creation and dissemination of highly realistic synthetic nude images exposes victims and their contacts to non-consensual toxic/pornographic content.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The harm was caused by human actors intentionally using deployed generative AI tools to create non-consensual deepfake pornographic images for blackmail.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes AI-generated content such as deepfakes, which falls under Risk Level 3 and requires transparency obligations under the EU AI Act.

AI system and alleged parties

• AI system: AI 'nudify' apps
• AI purpose: Deepfake Image Generation; Image Retouching
• Behaviour type: Tool
• Alleged developer: Unknown deepfake creators
• Alleged deployer: Unknown deepfake creators, Extortionists
• Alleged harmed parties: Women in India, Women, General public

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Minor
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Minor, indirect Minor
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Minor, indirect Minor
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Substantial

Malicious content

Reported: The report explicitly describes toxic or malicious content created or spread directly and indirectly by the incident.

Directly caused: AI-generated fake nude photos of Shefali were created and sent to her and five other friends/family members.

Indirectly caused: AI-generated nudes of a Bengaluru teen were shared in a boys' Instagram group.

Inferred additional harm: Thousands of non-consensual deepfake nude images are generated and circulated daily across platforms like Telegram.

Differential treatment

Reported: The report explicitly describes differential treatment of individuals or groups caused directly by the incident.

Directly caused: The 'undressing' tools are designed to pander to the male fantasy, specifically targeting and generating female-identified bodies even when male photos are uploaded.

Inferred additional harm: Systemic harassment and sexualization disproportionately targeting women and girls.

Civil rights

Reported: The report explicitly describes violations of human or civil rights caused directly and indirectly by the incident.

Directly caused: Violation of Shefali's right to privacy and personal dignity through the non-consensual creation and distribution of sexualized imagery.

Indirectly caused: Violation of the Bengaluru teen's rights through the distribution of fake nudes.

Inferred additional harm: Widespread violations of bodily autonomy and privacy rights for thousands of women targeted by these apps.

Privacy

Reported: The report explicitly describes privacy violations caused directly by the incident.

Directly caused: Loan apps obtained unauthorized access to Rishabh's phone gallery to harvest photos of his family.

Inferred additional harm: Mass privacy violations as malicious apps harvest personal data and photos from users' devices without meaningful consent.

Psychological

Reported: The report explicitly describes psychological and mental-health harm to people caused directly and indirectly by the incident.

Directly caused: Shefali experienced extreme distress and fear ('tears streaming across her face, fear and distress flashing in her eyes') upon receiving the fake nude photo.

Indirectly caused: Rishabh experienced shame and nervousness, preventing him from going to the police. The Bengaluru teen's daughter went through a 'rather difficult phase'.

Inferred additional harm: Widespread psychological trauma, anxiety, and social shame for numerous unnamed victims of these nudify apps.

Epistemic

Reported: The report explicitly describes epistemic harm caused directly by the incident.

Directly caused: The generation of fake nude photos that looked 'impossibly real', deceiving recipients into believing they were genuine photos of Shefali.

Inferred additional harm: Erosion of trust in visual evidence due to the proliferation of indistinguishable deepfakes.

Child sexual exploitation and abuse

Reported: The report explicitly describes a CSEA incident as reported by the source.

Indirectly caused: The Bengaluru teen shared AI-generated nudes of his minor classmate on Instagram.

Inferred additional harm: Proliferation of AI-generated child sexual abuse material (CSAM) as 1 in 10 minors reportedly knows peers who created deepfake nudes.

People affected

• Occurrences reported: 2
• People reportedly harmed: 8
• People reportedly exposed: 8

Potential causes

Management

• Growth Prioritization: Platforms prioritize user engagement over strict safety and moderation.
• Inadequate Risk Assessment: Lenders and platforms fail to mitigate the abuse potential of their systems.

Technology

• Generative Inpainting AI: AI models realistically replace clothed areas with generated nude pixels.
• Accessible Telegram Bots: Bots automate deepfake creation with simple photo uploads.
• Feminine Body Training Bias: Models are optimized to output female features regardless of input gender.

Data Inputs

• Unauthorized Gallery Access: Predatory loan apps exploit device permissions to harvest personal photos.
• Publicly Sourced Images: Social media and public photos are used without consent to generate fakes.
• Large Female Body Datasets: AI training relies on extensive datasets of clothed and nude female bodies.

Human Factors

• Malicious Extortion Motives: Loan sharks deliberately use deepfakes as leverage to blackmail victims.
• Social Malice and Bullying: Teens and individuals use deepfakes to damage reputations of peers.

Process and Methods

• Freemium Business Models: Free initial credits lower the economic barrier for malicious users.
• Flawed App Permission Vetting: App stores allow utility apps to request unnecessary gallery access.
• Reactive Takedown Processes: Takedowns occur after the psychological and social harm is already done.

Regulatory Environment

• Lenient Platform Policies: Telegram's lax content moderation allows illegal channels to thrive.
• Inadequate Legal Deterrents: Lack of swift prosecution fails to deter anonymous online blackmailers.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides clear, detailed accounts of specific incidents (Rishabh/Shefali and the Bengaluru teen), explains the underlying technology (inpainting), and details the platforms used (Telegram). The roles of the human actors and the AI tools are clearly distinguished.

The widespread availability of generative AI 'nudify' tools has enabled localized blackmail and harassment campaigns, particularly in India. While causing severe psychological harm and privacy violations to individuals, the incident represents a localized criminal threat manageable by standard law enforcement rather than a direct threat to national security.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: India
• Other affected: Global
• Alleged perpetrator: Predatory loan sharks and individual harassers

Threat characteristics

• Imminence: Long-term. The threat represents an ongoing, systemic capability concern regarding the misuse of generative AI rather than an immediate national security crisis.
• Autonomy: Human-controlled. The AI systems function as tools directly controlled and initiated by human operators to generate specific manipulated images.
• Novelty: Evolved capability. Represents an evolved capability where advanced generative AI inpainting tools are now easily accessible to non-technical malicious actors via Telegram.

Impact by dimension

• Physical security: Negligible. No physical systems, infrastructure, or human safety capabilities were compromised or targeted in this incident.
• Information security: Negligible. While deepfakes were generated, they were used for localized extortion and personal harassment rather than state-sponsored information warfare or intelligence operations.
• Sovereignty: Negligible. The incident does not involve threats to state authority, electoral processes, or core government decision-making.
• Economic security: Negligible. Financial loss was limited to individual extortion victims and does not threaten national financial systems or strategic technological competitive advantage.
• Societal stability: Minor. Widespread use of AI-nudify tools for blackmail and non-consensual sexual imagery causes severe psychological harm and privacy violations, primarily targeting women, but remains manageable under standard criminal law.