AI Transcription Tool Whisper Reportedly Inserting Fabricated Content in Medical Transcripts

OpenAI's AI-powered transcription tool Whisper, used to translate and transcribe audio content such as patient consultations with doctors, is advertised as having near “human level robustness and accuracy.” However, software engineers, developers and academic researchers have alleged that it is prone to making up chunks of text or even entire sentences and that some of the hallucinations can include racial commentary, violent rhetoric, and even imagined medical treatments.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The incident stems from Whisper's technical failure to perform reliably, resulting in a high rate of hallucinations and fabricated text during transcription.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Hospitals and medical centers are deploying Whisper in high-risk clinical environments despite explicit warnings from OpenAI against its use in such domains.
• 1.2 Exposure to toxic content: The model's hallucinations include generating violent rhetoric and unsolicited racial commentary, exposing users to inappropriate content.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The risk is caused by the Whisper AI model generating unexpected and erroneous hallucinations during its post-deployment operation.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The report describes Whisper being integrated into medical transcription tools used by clinicians in healthcare settings, which represents an AI system used in healthcare and diagnostic support.

AI system and alleged parties

• AI system: Whisper (OpenAI)
• AI purpose: Voice Recognition; Translation
• Behaviour type: Assistant
• Alleged developer: OpenAI
• Alleged deployer: OpenAI
• Alleged harmed parties: Patients reliant on Whisper, patients, Medical practitioners reliant on Whisper

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Minor
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Substantial
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes Whisper generating toxic content, including violent rhetoric and racial commentary.

Directly caused: Whisper directly generated toxic hallucinations, such as inventing a sentence claiming a speaker had a 'terror knife' and 'killed a number of people,' and adding unsolicited racial commentary.

Inferred additional harm: With Whisper being downloaded over 4.2 million times a month on HuggingFace, it is highly likely that many other users have been exposed to toxic or inappropriate hallucinations.

Differential treatment

Reported: The report explicitly describes how the Deaf and hard of hearing are at a disproportionate risk of harm from faulty transcriptions.

Indirectly caused: Deaf and hard of hearing individuals relying on Whisper-generated captions are unequally impacted because they cannot easily verify if the text has been fabricated.

Inferred additional harm: The widespread deployment of Whisper for closed captioning likely results in systemic unequal access to accurate information for the Deaf community.

Privacy

Reported: The report explicitly describes privacy concerns regarding the sharing of confidential medical consultations with tech vendors.

Indirectly caused: Patients' confidential medical consultations are being shared with third-party cloud systems like Microsoft Azure to facilitate Whisper-based transcription.

Inferred additional harm: It is likely that many of the patients whose 7 million medical visits were transcribed did not fully understand or explicitly consent to their sensitive medical data being processed by third-party AI systems.

Epistemic

Reported: The report explicitly describes epistemic harm through the generation of false information and fabricated medical terms.

Directly caused: Whisper directly generated false transcriptions, hallucinating in up to 80% of public meeting transcripts in one study and inventing non-existent medications like 'hyperactivated antibiotics.'

Inferred additional harm: The integration of Whisper into translation, search, and database systems likely spreads fabricated information across downstream digital ecosystems.

People affected

• Occurrences reported: 1
• People reportedly exposed: 7000000

Potential causes

Management

• Ignoring High-Risk Warnings: Hospitals deploy Whisper despite warnings against high-risk use.
• Prioritizing Speed Over Safety: Rushing integration of unverified tools to save administrative time.

Technology

• Frequent Model Hallucinations: Whisper generates fabricated text, including violent and racial commentary.
• Acoustic Vulnerabilities: Fabrications tend to occur during pauses, background sounds, or music.

Data Inputs

• Audio Ground Truth Erasure: Nabla erases original audio, preventing verification of transcript accuracy.
• Confidentiality and Privacy Risks: Sharing intimate medical audio with tech vendors raises privacy concerns.

Human Factors

• Overconfidence in AI Accuracy: Users trust Whisper's 'human-level' claims, ignoring potential errors.
• Lack of Manual Verification: Clinicians may fail to double-check AI transcripts against actual audio.

Process and Methods

• Deploying in High-Risk Domains: Using Whisper in medical transcription despite warnings against high-risk use.
• Inadequate Verification Processes: Lack of robust mechanisms to catch and correct hallucinated transcriptions.

Regulatory Environment

• Absence of AI Regulations: Lack of federal regulations governing AI safety and deployment standards.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide detailed, multi-sourced evidence from academic researchers, machine learning engineers, and former OpenAI employees, citing specific studies and statistics regarding Whisper's hallucination rates.
• Ambiguities identified: The exact technical cause of Whisper's high hallucination rate compared to other tools remains partially unexplained, though background noise and pauses are cited as factors.
• Alternative interpretations: The incident could be interpreted primarily as a user-error/overreliance issue (5.1) due to hospitals ignoring OpenAI's warnings, but the unique severity of Whisper's technical flaws supports a primary classification of lack of capability (7.3).

The widespread deployment of OpenAI's Whisper model in healthcare and public transcription, despite severe hallucination flaws and toxic output generation, poses minor national security concerns. The primary risks are localized to healthcare data integrity, patient safety vulnerabilities, and minor societal inequalities, which remain manageable under standard procedures.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Other affected: Unknown
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding AI reliability, model safety, and overreliance in high-risk sectors rather than an active, immediate crisis.
• Autonomy: Human-supervised. The AI tool acts as an assistant to transcribe audio, requiring human clinicians and users to verify and edit the output, though overreliance often limits active supervision.
• Novelty: Evolved capability. Speech-to-text is an established technology, but the high rate of sophisticated, fabricated hallucinations in advanced transformer-based models represents an evolved threat.

Impact by dimension

• Physical security: Minor. Whisper is integrated into healthcare, a critical infrastructure sector, transcribing millions of medical visits. While no physical harm is reported, fabricating medical terms introduces minor systemic risks to patient safety.
• Information security: Minor. The model generates toxic hallucinations, violent rhetoric, and racial commentary in transcripts. While not an active info-war campaign, it corrupts the integrity of public and private information ecosystems.
• Sovereignty: Negligible. No indication of threats to state authority, electoral systems, or core government decision-making processes.
• Economic security: Minor. Widespread deployment of flawed transcription software in commercial healthcare systems, coupled with deleting original audio, creates minor technological vulnerabilities in data verification.
• Societal stability: Minor. Presents minor societal risks including privacy concerns over patient data sharing and unequal impact on the Deaf and hard of hearing community who rely on potentially fabricated captions.