Washington State's Lottery AI Site Reportedly Generates Inappropriate User Image

An AI-powered website by Washington State's Lottery is reported to have inadvertently produced a softcore pornographic image of a user, leading to the site’s immediate shutdown out of caution.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.2 Exposure to toxic content

The AI system generated and exposed the user to inappropriate, sexually explicit content (a topless image of herself) which violates community norms regarding pornography.

Additional risk subdomains

• 7.3 Lack of capability or robustness: The AI system failed to perform robustly under deployment conditions, failing to adhere to the strict parameters set by developers to keep generated individuals fully clothed.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by an unexpected and unintended output generated by the AI system after it was deployed to the public.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: The system is an AI-powered image generator that creates synthetic images (AI-generated content), which falls under the Limited Risk category requiring transparency obligations.

AI system and alleged parties

• AI system: Washington's Lottery AI mobile site
• AI purpose: Image Generation; Deepfake Image Generation
• Behaviour type: Tool
• Alleged developer: Washington State's Lottery
• Alleged deployer: Washington State's Lottery

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the generation of inappropriate, sexually explicit content by the AI system.

Directly caused: The AI system generated a softcore pornographic image depicting the user's face on a topless body.

Psychological

Reported: The report explicitly describes psychological distress experienced by the affected user.

Directly caused: One user experienced shock and distress, describing the generated image of herself as 'disturbing to say the least' and stating she was 'completely shocked'.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 1

Potential causes

Management

• Inadequate Risk Assessment: Management failed to fully assess risks of public-facing generative AI tools.
• Delayed Site Takedown: Site remained online for days after the initial incident report was received.

Technology

• Generative Model Unpredictability: AI models can generate unintended NSFW outputs despite safety parameters.
• Inadequate Image Filtering: Built-in safety filters failed to detect and block the topless output.

Data Inputs

• User Headshot Uploads: Uploading arbitrary user faces combined with complex prompts caused anomalies.
• Unconstrained Prompt Inputs: Vacation themes like 'swim with sharks' triggered unexpected associations.

Human Factors

• Overreliance on AI Safeguards: Developers assumed pre-set rules would perfectly prevent inappropriate images.

Process and Methods

• Lack of Pre-deployment Testing: Testing did not uncover edge cases where safety parameters could be bypassed.
• Ineffective Output Validation: No real-time automated check to verify if generated images met clothing rules.

Regulatory Environment

• Lack of AI Safety Standards: No official regulations or standards governing public-facing AI deployments.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, consistent details about the incident, including direct quotes from the affected user and official statements from the Washington Lottery spokesperson confirming the event and subsequent website shutdown.
• Ambiguities identified: The specific AI model, platform, or third-party developers used to build the website were not disclosed.
• Alternative interpretations: None, as the facts of the generation and the subsequent shutdown are undisputed.

An AI-powered promotional website for the Washington State Lottery unintentionally generated a single sexually explicit image of a user. The incident represents a local software failure with negligible national security implications, and the site was promptly taken offline.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The website was taken offline immediately, resolving any immediate threat or ongoing operational risk.
• Autonomy: Human-controlled. The AI acted as a tool assisting the user based on direct inputs, though the generation step lacked real-time human review.
• Novelty: Established threat. Unintended generation of explicit content by AI image models is a well-documented and established technical vulnerability.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or human safety capabilities were compromised or targeted in this incident.
• Information security: Negligible. This was an isolated promotional website error, not a coordinated information warfare campaign, intelligence compromise, or state-sponsored deepfake operation.
• Sovereignty: Negligible. While the Washington State Lottery is a state agency, the incident was a minor promotional failure with no impact on core government operations or sovereignty.
• Economic security: Negligible. No strategic technology, critical supply chains, or financial systems were impacted or compromised by this promotional failure.
• Societal stability: Negligible. The incident caused individual psychological distress but did not involve mass surveillance, systematic discrimination, or large-scale threats to social cohesion.