Facebook translates 'good morning' into 'attack them', leading to arrest

Facebook's automatic language translation software incorrectly translated an Arabic post saying "Good morning" into Hebrew saying "hurt them," leading to the arrest of a Palestinian man in Beitar Illit, Israel.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The AI system failed to perform reliably under varying linguistic conditions, misinterpreting a common colloquial Arabic greeting as a violent threat.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Israeli police relied entirely on the automated translation output to make an arrest decision without verifying it with an Arabic-speaking officer.
• 1.3 Unequal performance across groups: The translation system exhibited lower accuracy and robustness for Arabic dialects compared to other languages, disproportionately exposing Arabic speakers to risk.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was triggered by an unexpected translation error produced by Facebook's deployed AI translation system.

EU AI Act risk tier

• Risk tier: 2 High Risk

Risk Level 2: High Risk. While translation is generally low risk, the system was utilized in a 'law enforcement application' to monitor social media and make arrest decisions, which has significant implications for safety and fundamental rights.

AI system and alleged parties

• AI system: None named
• AI purpose: Translation; Content Moderation
• Behaviour type: Tool
• Alleged developer: Facebook
• Alleged deployer: Facebook
• Alleged harmed parties: unnamed Palestinian Facebook user, Palestinian Facebook users, Facebook users, Arabic-speaking Facebook users

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Minor
• Civil rights: direct Negligible, indirect Minor
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Differential treatment

Reported: Yes, the reports describe how the translation error specifically targeted an Arabic-speaking individual.

Directly caused: The AI system's translation error specifically misidentified colloquial Arabic, translating a peaceful greeting into a threat.

Indirectly caused: The police acted on the automated translation without human verification, a practice that disproportionately impacts Arabic-speaking Palestinians under surveillance.

Civil rights

Reported: Yes, the reports explicitly describe the wrongful arrest and detention of the Palestinian worker.

Indirectly caused: The AI translation error directly led to the wrongful arrest, detention, and interrogation of Halawim Halawi, violating his civil rights.

Psychological

Reported: Yes, the reports describe the emotional distress and disruption caused to the man and his family due to the wrongful arrest.

Indirectly caused: The wrongful arrest, detention, and interrogation for several hours by police caused significant distress and anxiety to Halawim Halawi.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 1

Potential causes

Management

• Lack of AI Safety Guardrails: Facebook deployed automatic translation without real-world risk checks.
• Inadequate Risk Assessment: Management failed to assess translation risks in high-conflict zones.

Technology

• Neural Translation Flaws: Proprietary CNN/RNN translation model made a critical semantic error.
• Lack of Uncertainty Outputs: System did not convey translation confidence or alternative meanings.
• Dialect Translation Complexity: AI struggled with colloquial Arabic dialects vs standard Arabic.

Data Inputs

• Orthographic Similarity: Only one letter difference between good morning and verb to hurt.
• Informal Language Slang: Informal social media text is poorly mapped in translation training data.

Human Factors

• Automation Bias by Police: Officers relied entirely on automated translation without verification.
• Contextual Bias from Image: Posing with a bulldozer triggered suspicion due to historical attacks.

Process and Methods

• No Pre-Arrest Human Review: No Arabic-speaking officer read the post before the arrest was made.
• Flawed Predictive Policing: Police used social media monitoring algorithms to trigger direct arrests.

Regulatory Environment

• Policymaker Knowledge Gaps: Policymakers adopt automated tools without understanding their limitations.

Information quality

• Classification confidence: High
• Reason for confidence: The reports are highly consistent across multiple reputable news outlets, detailing the exact sequence of events, the specific mistranslation, and the subsequent actions of the police and Facebook.
• Ambiguities identified: None of significance.
• Alternative interpretations: The incident could be viewed primarily as a failure of police procedure rather than an AI failure, but the AI's translation error was the direct trigger.

A Palestinian worker was wrongfully arrested by Israeli police after Facebook's automatic translation software incorrectly translated 'good morning' into 'attack them' in Hebrew. The incident highlights the risks of overreliance on automated AI tools in law enforcement and intelligence monitoring without human-in-the-loop verification, resulting in a localized human rights violation.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: Israel
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding the reliability of AI translation tools in surveillance and law enforcement rather than an active crisis.
• Autonomy: Human-controlled. The AI acted as a translation tool to assist police, but human officers ultimately made the final decision to execute the arrest.
• Novelty: Evolved capability. Demonstrates an evolved risk where standard machine translation failures directly translate into physical law enforcement actions and wrongful detentions.

Impact by dimension

• Physical security: Negligible. No physical attacks, kinetic threats, or critical infrastructure damage occurred. The perceived threat was entirely a false positive caused by a translation error.
• Information security: Minor. Involved automated intelligence monitoring of social media, but the incident was a localized translation error rather than a coordinated disinformation campaign or intelligence compromise.
• Sovereignty: Minor. Minor impact on local law enforcement operations, where police temporarily acted on erroneous automated translation without manual verification before realizing the mistake.
• Economic security: Negligible. No impact on economic stability, strategic industries, or technological competitive advantage was reported.
• Societal stability: Minor. Led to the wrongful arrest and temporary detention of a Palestinian worker, representing a localized civil and human rights violation but not triggering wider societal instability.