The WHO's S.A.R.A.H. Bot Reported to Provide Inconsistent and Inadequate Health Information

The WHO's AI-powered health advisor, S.A.R.A.H. (Smart AI Resource Assistant for Health), is alleged to provide inconsistent and inadequate health information. The bot reportedly gives contradictory responses to the same queries, fails to offer specific contact details for healthcare providers, and inadequately handles severe mental health crises, often giving irrelevant or unhelpful advice.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The SARAH chatbot exhibits a clear lack of capability and robustness by providing contradictory answers, broken links, and geographically inappropriate crisis resources in a critical health context.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Users experiencing severe medical symptoms or mental health crises may unsafely rely on the chatbot instead of seeking professional medical care.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The risk of providing inconsistent and potentially dangerous health advice was an unexpected outcome of deploying the SARAH AI chatbot to assist the public.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system is a chatbot designed to interact with humans and provide health promotion advice, which falls under the category of chatbots requiring transparency.

AI system and alleged parties

• AI system: S.A.R.A.H.
• AI purpose: Health AI Assistant; Chatbot
• Behaviour type: Assistant
• Alleged developer: WHO
• Alleged deployer: WHO, S.A.R.A.H. (Smart AI Resource Assistant for Health)
• Alleged harmed parties: People seeking medical advice, General public

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Differential treatment

Reported: The report explicitly describes differential treatment in the utility of crisis support based on geographic location.

Directly caused: The chatbot provided the US National Suicide Prevention Lifeline to users experiencing suicidal ideation, which is not useful for users outside of the United States.

Inferred additional harm: Non-US users in crisis receive lower quality and potentially useless support compared to US-based users, creating a geographic disparity in safety.

Epistemic

Reported: The report explicitly describes epistemic harm in the form of inconsistent, contradictory, and poor-quality health answers.

Directly caused: The chatbot frequently gave contradictory answers to the same queries, failed to provide contact details of healthcare providers, and provided broken links.

Inferred additional harm: Widespread use of the bot could mislead users about medical facts, healthy living, and available healthcare resources.

People affected

• Occurrences reported: 1

Potential causes

Management

• Premature Deployment: WHO released the bot despite critical shortcomings in mental health advice.

Technology

• Inconsistent LLM Outputs: The bot provides contradictory answers to identical queries.
• Hallucinated or Broken Links: The chatbot dispenses broken links and poor-quality answers.
• Context Switching Failure: The bot abruptly switches from provider search to tobacco cessation.

Data Inputs

• US-Centric Crisis Data: Suicide helpline advice defaults to US-only numbers for global users.
• Incomplete Provider Directory: The system failed to retrieve or share local provider contact details.

Human Factors

• User Reliance on Critical Advice: Users seeking mental health support receive inadequate global resources.

Process and Methods

• Inadequate Testing and Validation: The chatbot was deployed with critical flaws in crisis response.

Regulatory Environment

• Lack of Chatbot Health Standards: No clear regulatory standards govern medical advice provided by chatbots.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly outlines the specific failures of the SARAH chatbot, including contradictory answers and geographic limitations of its crisis support. While it lacks quantitative data on actual user harm, the nature of the AI's limitations is well-documented by testing.
• Ambiguities identified: The exact number of users who interacted with the bot or experienced harm is not specified.
• Alternative interpretations: The system could be viewed as a simple informational tool (Level 4) rather than a chatbot with moderate risk (Level 3), but its role in crisis support elevates its risk profile.

The World Health Organization's AI health advisor, SARAH, exhibited significant reliability issues, including providing contradictory health advice and geographically inappropriate crisis hotlines. While this presents public health safety concerns, it does not pose a direct threat to national security, critical infrastructure, or sovereignty.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Global audience
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing public health communication issue rather than an immediate national security crisis.
• Autonomy: Full autonomy. The chatbot generates and delivers responses directly to users in real-time without human-in-the-loop verification.
• Novelty: Established threat. Inconsistencies, hallucinations, and geographic limitations in LLM-based chatbots are well-established technological limitations.

Impact by dimension

• Physical security: Negligible. No physical security threats, kinetic attacks, or critical infrastructure compromise occurred as a result of this chatbot deployment.
• Information security: Negligible. No evidence of state-sponsored disinformation, intelligence compromise, or systematic information warfare targeting national security systems.
• Sovereignty: Negligible. No disruption to core government operations, electoral systems, or sovereign decision-making processes.
• Economic security: Negligible. No strategic technology theft, financial system attacks, or critical supply chain disruptions occurred.
• Societal stability: Minor. Public health advisor failures, including providing ineffective international crisis resources, could lead to localized safety risks, but do not threaten overall societal stability.