Google Lens’s Camera-Based Translation Feature Provided an Offensive Mistranslation of a Book Title in Korean

A book title by Korea’s first minister of culture was mistranslated into an offensive phrase by Google Lens’s camera-based translation feature allegedly due to its training on internet communications and a lack of context.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.2 Exposure to toxic content

The AI system generated and exposed the user to a highly offensive and profane sexual slur, violating community norms regarding appropriate content.

Additional risk subdomains

• 7.3 Lack of capability or robustness: The system failed to perform robustly under distributional shift, failing to understand the context of a formal book title versus internet slang.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The mistranslation was caused by the Google Lens AI system's failure to generalize context post-deployment, which was an unintended outcome of its translation goal.

EU AI Act risk tier

• Risk tier: 4 Minimal or No Risk

Minimal or No Risk: The system is a general-purpose translation tool used for everyday text translation, which poses low or negligible risk to safety or fundamental rights under the EU AI Act.

AI system and alleged parties

• AI system: None named
• AI purpose: Translation; Image Description
• Behaviour type: Tool
• Alleged developer: Google
• Alleged deployer: Google
• Alleged harmed parties: Google Lens users

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 1

Potential causes

Management

• Premature Feature Scaling: Prioritizing rapid language rollout over robust verification.
• Lack of Human-in-the-Loop: Deploying translation without community review oversight.

Technology

• Generalization Challenge: Models struggle to perform beyond engineered environments.
• Lack of Contextual Awareness: AI cannot resolve subtext or differentiate literal from slang.

Data Inputs

• Internet Slang Training Data: Training on web forums leads to offensive slang translations.
• Lack of Labeled Context: Missing metadata about text genre or source style.

Human Factors

• Overreliance on Automation: Assuming machine translation is robust across low-resource languages.

Process and Methods

• Lack of Translation Validation: No systematic verification of output quality before scaling.
• Missing Feedback Loop: No immediate way for users to report or correct errors.
• No Source Text Comparison: Failure to provide links back to original untranslated text.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides clear, direct evidence of the translation error, including side-by-side images of the book cover and the resulting translation. The linguistic explanation for why the error occurred is detailed and logical, leaving little ambiguity about the nature of the incident.

A consumer-facing translation tool (Google Lens) mistranslated a Korean book title into an offensive English slur due to training data bias and lack of context. The incident represents a minor post-deployment software bug with negligible implications for national security.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: No clear primary
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. This is a post-deployment translation bug and does not represent an active national security crisis.
• Autonomy: Human-controlled. The AI system operates as a tool, performing translation on direct user input without independent action.
• Novelty: Established threat. Machine translation errors and offensive outputs due to training data bias are well-documented, established issues.

Impact by dimension

• Physical security: Negligible. No physical systems, infrastructure, or human safety were threatened by this consumer translation error.
• Information security: Negligible. This was an unintentional consumer translation error, not an active information warfare campaign or intelligence compromise.
• Sovereignty: Negligible. No impact on state authority, territorial control, or core government functions was reported.
• Economic security: Negligible. No strategic technology theft or critical economic infrastructure was affected by this translation failure.
• Societal stability: Negligible. While the translation generated an offensive slur, it was an isolated, unintentional incident with no impact on societal stability or human rights.