Google Healthcare AI Model Med‑Gemini Allegedly Produces Non‑Existent 'Basilar Ganglia' Term in Published Output

Google’s Med‑Gemini healthcare AI reportedly produced the non‑existent term "basilar ganglia" in public launch materials, conflating two distinct brain structures. The error reportedly appeared in both a blog post and an arXiv preprint. Google is reported to have initially edited the blog without acknowledgment, later calling it a typo.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The AI systems exhibited a lack of robustness by hallucinating non-existent medical terms and producing inconsistent diagnostic outputs based on minor prompt variations.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Experts highlight the severe risk of automation bias, where clinicians may over-rely on credible-sounding AI outputs and fail to catch critical medical errors.
• 3.1 False or misleading information: The AI models generated false medical terms and incorrect diagnoses that were published in research papers and promotional materials.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The medical errors and hallucinations were generated unintentionally by the AI systems during testing and pilot deployment phases.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The AI systems are diagnostic tools used in healthcare, which is explicitly classified as a high-risk application under the EU AI Act.

AI system and alleged parties

• AI system: Med-Gemini, MedGemma (Google)
• AI purpose: Medical Diagnosis Support; Technical Text Generation
• Behaviour type: Assistant
• Alleged developer: Google
• Alleged deployer: Google
• Alleged harmed parties: Patients whose care could be affected by undetected AI hallucinations, Medical research community citing or using erroneous AI outputs, Healthcare professionals relying on AI‑generated diagnostic content

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Epistemic

Reported: The report explicitly describes epistemic harm in the form of AI-driven medical hallucinations and inconsistent diagnostic fabrications published in research materials.

Directly caused: Med-Gemini fabricated the non-existent anatomical term 'basilar ganglia' in a published pre-print research paper and promotional blog post.

Indirectly caused: MedGemma generated false negative diagnoses (e.g., calling an abnormal rib X-ray normal) and multiple hallucinated diagnoses depending on prompt phrasing.

Inferred additional harm: Medical professionals reading the uncorrected research paper or blog post may have been misinformed about the model's true capabilities and accuracy.

People affected

• Occurrences reported: 1

Potential causes

Management

• Rapid Adoption of Immature Tech: Management pushed immature models into trusted tester and clinical pilots.
• Downplaying Clinical Risks: Google downplayed a critical medical hallucination as a simple misspelling.

Technology

• AI Model Hallucination: Model conflated basal ganglia and basilar artery into a nonexistent area.
• Phrasing Sensitivity: MedGemma missed pathologies when queries were simplified or phrased differently.
• Training Data Error Propagation: Model learned and repeated common mis-transcriptions present in training data.

Data Inputs

• Inaccurate Training Labels: Training data contained mis-transcriptions that the model learned as correct.
• Erroneous Clinical Notes: Incorrect assessments in notes are read and propagated further by the AI.

Human Factors

• Automation Bias: Complacency leads clinicians to miss errors in generally accurate systems.
• Reviewer Oversight: Over 50 authors and medical reviewers failed to catch the clinical error.

Process and Methods

• Missing Confabulation Alerts: Systems lack real-time hallucination detection to warn clinicians of errors.
• Flawed Review Workflows: Designing workflows where humans review highly hallucinated drafts is inefficient.
• Non-Transparent Error Correction: Google quietly edited the blog post without updating the pre-print paper.

Regulatory Environment

• Wild West Regulatory Stage: Healthcare AI lacks clear standards and high bars of evidence for deployment.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides clear, detailed accounts of specific errors made by Google's Med-Gemini and MedGemma models, supported by medical experts and researchers who documented the issues. The developer's response is also included, confirming the occurrence of the 'basilar ganglia' error.
• Ambiguities identified: Whether the 'basilar ganglia' error was a pure hallucination or a simple transcription typo, though experts strongly lean toward hallucination.
• Alternative interpretations: Google downplayed the error as a simple misspelling or typo, whereas medical experts interpreted it as a dangerous hallucination demonstrating model limitations.

Google's Med-Gemini and MedGemma healthcare models demonstrated diagnostic errors and hallucinations during testing. While highlighting important safety concerns regarding automation bias in clinical settings, the incident remains a commercial software quality issue with negligible national security implications.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: No clear primary
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The issues represent ongoing, long-term technical challenges in AI development rather than an active national security crisis.
• Autonomy: Human-supervised. The AI models are designed to assist medical professionals, requiring human review and verification before final clinical decisions are made.
• Novelty: Established threat. Large language model hallucinations and sensitivity to prompt engineering are well-known, established technical limitations.

Impact by dimension

• Physical security: Negligible. The incident involves commercial medical AI models in testing phases with no reported physical injuries, deaths, or critical infrastructure disruptions.
• Information security: Negligible. No intelligence compromise, classified data exposure, or state-sponsored disinformation campaigns are associated with this incident.
• Sovereignty: Negligible. The diagnostic errors of a commercial healthcare AI do not impact state authority, border controls, or government decision-making.
• Economic security: Negligible. Model limitations during pilot testing do not pose a threat to national economic stability or strategic technological competitiveness.
• Societal stability: Negligible. There is no evidence of mass surveillance, systemic discrimination, or threats to civil liberties resulting from these testing errors.