Kenyan Data Annotators Allegedly Exposed to Graphic Content for OpenAI's AI

Sama AI's Kenyan contractors were reportedly asked with excessively low pay to annotate a large volume of disturbing content to improve OpenAI's generative AI systems such as ChatGPT, and whose contract was terminated prior to completion by Sama AI.

Risk classification

• Primary risk domain: 6 Socioeconomic & Environmental
• Primary risk subdomain: 6.2 Increased inequality and decline in employment quality

The incident highlights the exploitation of low-wage workers in the Global South, who faced hazardous and traumatic working conditions to support high-value AI development.

Additional risk subdomains

• 1.2 Exposure to toxic content: Workers were exposed to highly toxic, graphic, and illegal content, including child sexual abuse and violence, to train the AI safety filters.

Causal factors

• Entity: Human
• Intent: Unintentional
• Timing: Pre-deployment

The severe psychological trauma experienced by the workers was caused by human organizational decisions regarding outsourcing and labor management, and was an unintentional byproduct of the safety training process.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: ChatGPT is a chatbot and generative AI system, which falls under Risk Level 3 requiring specific transparency obligations so users know they are interacting with an AI.

AI system and alleged parties

• AI system: AI-powered safety mechanism, ChatGPT (OpenAI)
• AI purpose: Content Moderation; Chatbot
• Behaviour type: Assistant
• Alleged developer: OpenAI
• Alleged deployer: OpenAI
• Alleged harmed parties: Kenyan Sama AI employees

Harm severity

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Minor, indirect Minor
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Substantial, indirect Negligible

Differential treatment

Reported: The reports describe the exploitation of low-income workers in the Global South (Kenya) who were paid a fraction of what US workers would receive for the same traumatic work.

Directly caused: Kenyan workers were paid between $1.32 and $2 per hour, which is a small fraction of US minimum wage, to perform highly traumatic content moderation that OpenAI outsourced to the Global South.

Indirectly caused: N/A

Inferred additional harm: This reflects a systemic pattern where tech companies outsource hazardous and traumatic data labeling tasks to low-income countries to minimize costs, leading to unequal safety and labor standards.

Civil rights

Reported: The reports describe inhumane working conditions and labor exploitation.

Directly caused: Sama and OpenAI subjected Kenyan workers to exploitative working conditions, paying them less than $2 per hour for highly traumatic work without providing adequate mental health support.

Indirectly caused: N/A

Inferred additional harm: The lack of regulatory oversight in Kenya for digital microwork likely allowed these exploitative labor practices to continue without legal recourse for the affected workers.

Psychological

Reported: The reports explicitly describe severe psychological distress, trauma, PTSD, and nightmares among the Kenyan data annotators.

Directly caused: At least 50 Kenyan workers suffered from severe mental trauma, recurring nightmares, anxiety, and social withdrawal after reading graphic descriptions of child sexual abuse, bestiality, and torture.

Indirectly caused: One worker's marriage collapsed due to his severe emotional distance and psychological trauma resulting from the work.

Inferred additional harm: It is likely that many of the 50 workers continue to suffer from long-term PTSD and psychological distress without access to adequate professional mental health support.

Child sexual exploitation and abuse

Reported: The reports explicitly mention that workers had to read and label graphic descriptions of child sexual abuse (CSAM/CSEA) and that a pilot project involved collecting C4 (child sexual abuse) images.

Directly caused: Sama workers were forced to read and label graphic text descriptions of child sexual abuse, and in a pilot project, collected 1,400 images that included C4 (child sexual abuse) category material.

Indirectly caused: N/A

Inferred additional harm: N/A

People affected

• Occurrences reported: 1
• People reportedly harmed: 50
• People reportedly exposed: 50

Potential causes

Management

• Outsourcing Risk Management: OpenAI offloaded mental health risks to third-party contractor Sama.
• Prioritizing Cost Over Safety: Paying low wages to maximize profit margins on AI development.
• Inadequate Oversight of Wellness: OpenAI relied on Sama's wellness claims without verifying actual support.

Technology

• Toxic Pre-training Datasets: GPT-3 trained on internet data containing extreme toxicity and bias.
• Lack of Automated Filtering: AI models cannot filter or label toxic data without human intervention.
• Complex Safety Filter Needs: Building safety filters requires feeding AI graphic examples of harm.

Data Inputs

• Graphic Text Snippets: Workers read thousands of detailed descriptions of abuse and violence.
• Illegal Image Datasets: Pilot project included child sexual abuse images illegal under US law.
• Ambiguous Content Nuance: Edge cases like ambiguous consent make labeling difficult to standardize.

Human Factors

• Severe Psychological Trauma: Exposure to graphic content caused PTSD, nightmares, and social isolation.
• Worker Vulnerability: High unemployment in Kenya forced workers to accept low-paying, toxic work.
• Inadequate Counseling Access: High productivity demands prevented workers from accessing 1:1 therapy.

Process and Methods

• High Productivity Demands: Workers pressured to label up to 250 passages per nine-hour shift.
• Poor Communication Channels: Miscommunication led to accidental collection of illegal C4 images.
• Inadequate Task Vetting: Sama did not vet the image pilot request through proper channels.

Regulatory Environment

• Absence of AI Labor Regulations: No clear laws governing digital microwork or content moderation in Kenya.
• No Minimum Wage Enforcement: Lack of local minimum wage laws enabled low pay for traumatic work.
• Lack of Global Safety Standards: No international standards protect outsourced AI data annotators.

Information quality

• Classification confidence: High
• Reason for confidence: The reports are highly detailed, based on an in-depth investigation by TIME magazine, including interviews with multiple workers, payslips, and official statements from both OpenAI and Sama. The facts regarding the working conditions, wages, and psychological impact are consistent across all sources.
• Ambiguities identified: There is some minor disagreement between Sama and the workers regarding the exact number of text passages expected to be labeled per shift (70 vs 150-250) and the availability of 1-on-1 counseling.
• Alternative interpretations: None. The primary event is clearly an issue of labor exploitation and psychological harm in the data-labeling supply chain of AI development.

The incident involves OpenAI outsourcing traumatic content moderation to Kenyan workers under exploitative conditions. While raising significant ethical, labor, and human rights concerns regarding the global AI supply chain, its direct impact on national security remains minor.

• Overall national security impact: Minor
• Response level: Moderate
• Scope: Multiple nations
• Primary target: Kenya
• Other affected: United States
• Alleged perpetrator: OpenAI

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding AI supply chain ethics and labor standards rather than an active crisis.
• Autonomy: Human-controlled. The incident involves human data annotators manually labeling text and images to train filters, with no autonomous AI actions.
• Novelty: Evolved capability. Represents an evolved capability of existing content moderation labor exploitation issues, now applied to generative AI safety alignment.

Impact by dimension

• Physical security: Negligible. No physical security threats, kinetic attacks, or critical infrastructure compromise occurred during this data-labeling operation.
• Information security: Negligible. No intelligence capabilities were compromised, and no state-sponsored information warfare operations were conducted.
• Sovereignty: Negligible. No threats to state authority, territorial control, or core government operations were identified in this commercial outsourcing dispute.
• Economic security: Minor. Highlights minor supply chain vulnerabilities in AI development, specifically the reliance on outsourced labor for safety training, but caused no direct threat to strategic industries.
• Societal stability: Minor. Involved exploitation and psychological trauma of outsourced data annotators in Kenya, representing minor localized human rights and labor concerns within the AI supply chain.