Stable Diffusion Exhibited Biases for Prompts Featuring Professions

Stable Diffusion reportedly posed risks of bias and stereotyping along gender and cultural lines for prompts containing descriptors and professions.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.1 Unfair discrimination and misrepresentation

Stable Diffusion's image generation systematically underrepresents women and people of color in high-paying professions while overrepresenting them in low-paying and criminal categories, amplifying real-world stereotypes.

Additional risk subdomains

• 1.3 Unequal performance across groups: The reports note that related AI tools like commercial facial-recognition and gender identification systems perform with lower accuracy for women of color.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The bias and amplification of stereotypes are generated by the deployed Stable Diffusion model, representing an unintentional outcome of training on uncurated internet data.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: Stable Diffusion is a generative AI system capable of producing synthetic image content and deepfakes, which under the EU AI Act are subject to transparency obligations to ensure users are aware they are interacting with AI-generated media.

AI system and alleged parties

• AI system: Stable Diffusion (Stability AI)
• AI purpose: Image Generation; Visual Art Generation
• Behaviour type: Tool
• Alleged developer: Stability AI, Runway, LAION, EleutherAI, CompVis LMU
• Alleged deployer: Stability AI
• Alleged harmed parties: Women, racial minority groups, gender minority groups

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Minor
• Differential treatment: direct Minor, indirect Minor
• Civil rights: direct Negligible, indirect Minor
• Democracy: direct Minor, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Substantial
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the generation of biased and stereotypical content, though it does not focus on toxic or malicious content in the traditional sense.

Indirectly caused: The report notes that Stable Diffusion has been used by third parties to generate pornographic and violent content due to its permissive and unregulated platform policies.

Differential treatment

Reported: The report explicitly describes systematic differential treatment and representation of demographic groups in the generated images.

Directly caused: Stable Diffusion's image generation disproportionately associates high-paying jobs (like judge, CEO, lawyer) with lighter-skinned men, while overrepresenting darker-skinned individuals and women in low-paying jobs (like fast-food worker, housekeeper) and criminal categories.

Indirectly caused: The amplification of these stereotypes could lead to professional and educational barriers, reinforcing the perception that certain groups do not belong in high-status roles.

Inferred additional harm: As these biased models are integrated into commercial software (like Adobe and Canva), they may perpetuate systemic discrimination in marketing, media, and corporate environments at scale.

Civil rights

Reported: The report explicitly describes potential violations of civil rights, particularly in the context of law enforcement and policing.

Indirectly caused: The report warns that using biased text-to-image AI for suspect sketching in policing could exacerbate racial bias in the criminal justice system and lead to wrongful convictions.

Inferred additional harm: The widespread deployment of biased generative models in public and private sectors could systematically undermine equal opportunity and civil rights protections for women and minorities.

Democracy

Reported: The report explicitly describes the use of AI-generated images in political campaigns to influence public opinion.

Directly caused: The Republican National Committee used AI-generated images in an anti-Biden political ad depicting a group of mostly White border agents apprehending 'illegals', reaching close to a million people on social media.

Inferred additional harm: The proliferation of highly realistic, biased, or fabricated AI images in political discourse could erode public trust, manipulate voter perceptions, and degrade democratic debate.

Epistemic

Reported: The report explicitly describes epistemic harm through the generation of highly realistic images that distort reality and are difficult to distinguish from real photographs.

Directly caused: Stable Diffusion generates images that present a distorted, highly stereotyped version of the world (e.g., depicting almost no women engineers or judges), which users may mistake for objective representations.

Indirectly caused: The report warns of a 'snowball effect' where biased AI-generated images are posted online, subsequently ingested as training data for future models, compounding the distortion of reality.

Inferred additional harm: The mass accumulation of AI-generated distortions online could permanently pollute the digital information ecosystem, making it increasingly difficult for the public to maintain a shared consensus reality.

People affected

• Occurrences reported: 1
• People reportedly exposed: 125000000

Potential causes

Management

• Laissez-faire Philosophy: Management prioritized decentralized AI over implementing safety controls.
• Layoffs of AI Ethicists: Corporate layoffs of ethics teams as market competition intensified.

Technology

• Algorithmic Amplification: Model hones in on dominant categories, magnifying dataset inequalities.
• Lack of Built-in Mitigation: No de-biasing mechanisms baked into the model training or evaluation.

Data Inputs

• Biased Training Dataset: LAION-5B contains uncurated images scraped from the biased internet.
• US and English Centricity: Dataset is dominated by US data and English-linked images.

Human Factors

• Perceived Objectivity of AI: Users trust AI outputs as objective, legitimizing encoded biases.
• Unchecked Deployment in Police: Police departments adopt biased AI tools without oversight.

Process and Methods

• Focus on Scale over Safety: Industry focus on bigger models and faster releases over safeguards.
• Lack of Dataset Curation: Images collected programmatically without human curation or filtering.

Regulatory Environment

• Absence of AI Safeguards: Lack of government regulations to enforce bias mitigation in AI.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide extensive, detailed empirical analyses of Stable Diffusion's outputs across thousands of generated images, using established scientific methodologies like the Fitzpatrick Skin Scale and comparisons with official US Bureau of Labor Statistics data. The findings are consistent across multiple independent investigations (Bloomberg, Hugging Face, Gizmodo), leaving very little ambiguity about the model's behavior and biases.
• Ambiguities identified: The exact proportion of training data from different countries in the LAION-5B dataset is not fully quantified, and the potential future harms in law enforcement are described as warnings rather than documented occurrences.
• Alternative interpretations: One could argue the bias is purely a reflection of internet data rather than a failure of the model itself, though the reports show the model actively amplifies these disparities beyond real-world statistics.

An analysis of Stable Diffusion revealed systematic amplification of racial and gender stereotypes in image generation. While presenting significant ethical and societal concerns regarding bias and minor implications for political information manipulation, the incident does not pose an immediate or substantial threat to national security.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Global
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The issue represents an ongoing, long-term societal and technological concern rather than an immediate crisis requiring urgent intervention.
• Autonomy: Human-controlled. The AI operates strictly as a tool, generating images based on direct text prompts provided by human users.
• Novelty: Evolved capability. Algorithmic bias is an established threat, but the scale and realistic nature of generative text-to-image models represent an evolved capability.

Impact by dimension

• Physical security: Negligible. No physical security threats or critical infrastructure impacts were reported or observed in connection with this image generation model.
• Information security: Minor. AI-generated images were utilized in domestic political campaigning to influence public opinion, showing minor potential for automated propaganda.
• Sovereignty: Negligible. No direct compromise of state authority, although experts warned about potential future risks if biased tools are adopted in law enforcement.
• Economic security: Negligible. No threats to financial systems, strategic industries, or technological competitive advantages were reported.
• Societal stability: Minor. The model amplifies demographic disparities and stereotypes at scale, potentially reinforcing societal inequalities, though it does not represent state-level oppression.