Stable Diffusion Allegedly Used Artists' Works without Permission for AI Training

Text-to-image model Stable Diffusion was reportedly using artists' original works without permission for its AI training.

Risk classification

• Primary risk domain: 6 Socioeconomic & Environmental
• Primary risk subdomain: 6.3 Economic and cultural devaluation of human effort

The training of AI models on copyrighted artworks without consent and the subsequent generation of style-replicating images directly devalues human creative effort and threatens the livelihoods of artists.

Additional risk subdomains

• 1.2 Exposure to toxic content: Lensa AI was reported to generate highly sexualized and topless images of users and celebrities without their consent.
• 2.1 Compromise of privacy by obtaining, leaking or correctly inferring sensitive information: The LAION-5B dataset used to train the models was found to contain private medical records and personal photos scraped without consent.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident stems from intentional human decisions by developers to scrape copyrighted images without consent and commercialize models trained on that data, leading to post-deployment harms.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes AI-generated content and deepfakes (such as Lensa's magic avatars and Stable Diffusion's image generation), which fall under Limited Risk due to transparency obligations to ensure users are aware they are interacting with AI-generated content.

AI system and alleged parties

• AI system: Lensa AI, Midjourney, Stable Diffusion (Midjourney, Prisma Labs, Stability AI)
• AI purpose: Visual Art Generation; Image Generation
• Behaviour type: Assistant
• Alleged developer: Stability AI, Runway, Lensa AI, LAION, EleutherAI, CompVis LMU
• Alleged deployer: Stability AI, Midjourney, Lensa AI, DeviantArt
• Alleged harmed parties: digital artists, artists publishing on social media, artists

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Minor, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Minor
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Substantial, indirect Negligible
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: The report explicitly describes financial losses to artists due to siphoned commissions and reduced work requests, though specific total dollar amounts of losses are not quantified.

Directly caused: Artists like Greg Rutkowski reported receiving far fewer requests from authors for book covers since the release of AI generators, directly impacting their income.

Inferred additional harm: It is highly likely that thousands of commercial artists, illustrators, and photographers have experienced significant aggregate financial losses due to lost commissions and downward pressure on pricing.

Malicious content

Reported: The report explicitly describes the creation of toxic and malicious content, including nonconsensual sexual imagery and alt-right political edits.

Directly caused: Lensa AI was used to generate topless and highly sexualized avatars of users and celebrities without their consent, bypassing safety filters.

Indirectly caused: Sarah Andersen's simple comic style made it easy for malicious actors on 4chan to manually edit her work to promote neo-Nazi ideology, creating a 'shadow self' that harassed her.

Inferred additional harm: It is likely that many other individuals have been targeted with nonconsensual deepfake pornography or harassing imagery generated using these tools.

Differential treatment

Reported: The report explicitly describes algorithmic bias and differential treatment based on physical characteristics.

Directly caused: Lensa AI exhibited demographic bias by lightening the skin of non-white users, altering facial features to look more Caucasian, and slimming the bodies of plus-sized users like Yasemin.

Inferred additional harm: Other users of color or plus-sized individuals likely experienced similar discriminatory or exclusionary outputs from the app's default training biases.

Civil rights

Reported: The report describes potential violations of the right of publicity and privacy rights under GDPR and California law.

Directly caused: The unauthorized use of artists' names and unique styles in commercial AI prompts is alleged to violate their right of publicity.

Privacy

Reported: The report explicitly describes privacy violations, including the scraping of private medical records and personal photos.

Directly caused: The LAION-5B dataset used to train Stable Diffusion was found to contain thousands of private patient medical record photos and individuals' faces without their consent.

Inferred additional harm: It is likely that many other individuals had private or sensitive personal data scraped from the internet and incorporated into the training models without their knowledge.

Psychological

Reported: The report explicitly describes psychological distress and feelings of violation among affected artists and users.

Directly caused: Artists like Sarah Andersen and Karla Ortiz reported feeling 'violated' and experiencing a 'violation of identity' upon discovering their life's work was used to train AI models without consent.

Inferred additional harm: It is likely that thousands of other artists whose works were scraped without consent experienced similar feelings of distress, violation, and anxiety regarding their career longevity.

Epistemic

Reported: The report explicitly describes concerns about the erosion of public trust and the inability to distinguish real images from AI-generated fakes.

Directly caused: Stable Diffusion was used to generate realistic-looking combat photos of the Ukraine war featuring fake Getty Images watermarks, demonstrating how easily the tool can fabricate photojournalistic evidence.

Inferred additional harm: The proliferation of highly realistic AI-generated images is likely to cause widespread epistemic confusion, making it difficult for the public to verify real-world events.

People affected

• Occurrences reported: 1
• People reportedly harmed: 10
• People reportedly exposed: 1000000

Potential causes

Management

• Prioritizing Profit Over Ethics: Companies monetized open-source models trained on non-consensual data.
• Inadequate Risk Assessment: Management failed to assess copyright and safety risks before launch.
• Lack of Corporate Accountability: DeviantArt adopted Stable Diffusion despite violating community trust.

Technology

• Diffusion Model Lossy Compression: Denoising reconstructs and interpolates latent images of training data.
• NSFW Filter Vulnerabilities: Users bypassed safety filters to generate non-consensual sexual imagery.
• Style Mimicry Capabilities: Algorithms mimic specific artists' styles using their names as prompts.

Data Inputs

• Non-Consensual Data Scraping: LAION-5B dataset scraped billions of internet images without artist consent.
• Inclusion of Sensitive Data: Dataset included copyrighted works, watermarks, and private medical records.
• Lack of Input Verification: No verification of ownership or rights for images used in AI training.

Human Factors

• User Prompt Exploitation: Users intentionally input prompts to bypass safety filters or copy styles.
• Disregard for Artist Rights: AI developers prioritized technical capabilities over ethical data sourcing.
• Consumer Demand for Cheap Art: Public viral adoption of cheap AI portraits ignored ethical implications.

Process and Methods

• Opt-Out Instead of Opt-In: Artists were default opted-in to training datasets with no initial control.
• Inadequate Content Moderation: Platforms failed to filter out infringing or harmful generated outputs.
• Lack of Source Attribution: No system to credit or compensate artists whose work trained the models.

Regulatory Environment

• Ambiguous Copyright Laws: Current laws fail to address AI training on scraped data.
• Lack of AI Safety Standards: No regulatory framework governing generative AI deployment and data sourcing.
• Delayed Legal Enforcement: Lawsuits are reactive, leaving artists without immediate legal recourse.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide comprehensive coverage of the incident from multiple perspectives, including first-hand accounts from affected artists, legal analyses of the lawsuits, technical explanations of diffusion models, and statements from the AI companies. The facts regarding data scraping, the class-action lawsuit, and the specific harms are well-documented and consistent across sources.
• Ambiguities identified: The exact legal status of fair use regarding AI training inputs remains unresolved in the courts at the time of the reports.
• Alternative interpretations: Some proponents view the technology as a transformative tool similar to the invention of photography, rather than a parasitic or infringing system.

The viral adoption of generative AI models trained on scraped data sparked widespread legal disputes over copyright and privacy. While the incident highlights significant challenges to intellectual property frameworks, privacy rights, and potential information manipulation, its direct national security implications remain minor and manageable within standard legal and regulatory procedures.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: United Kingdom, European Union
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The incident represents ongoing strategic, legal, and regulatory concerns regarding intellectual property and data privacy rather than an active crisis.
• Autonomy: Human-controlled. The AI systems function as creative assistants, generating outputs based on direct human prompts, inputs, and selections.
• Novelty: Evolved capability. Represents a significant advancement in the scale, accessibility, and sophistication of generative image models compared to prior technologies.

Impact by dimension

• Physical security: Negligible. No physical security threats, kinetic attacks, or critical infrastructure compromise were reported in connection with this incident.
• Information security: Minor. Stable Diffusion was used to generate realistic-looking combat photos of the Ukraine war, demonstrating potential for information manipulation, though no coordinated state-sponsored campaign was identified.
• Sovereignty: Negligible. No direct threats to state authority, electoral systems, or core government decision-making processes were reported.
• Economic security: Minor. Notable economic impact on the creative industry, including loss of commissions for artists and legal disputes over copyright, but does not threaten strategic national industries.
• Societal stability: Minor. Privacy violations occurred via unauthorized scraping of private medical records, alongside demographic bias and nonconsensual sexualized imagery generation, which are manageable under standard legal frameworks.