Racial Bias in Lung Function Diagnostic Algorithm Leads to Underdiagnosis in Black Men

A study published in JAMA Network Open reveals that racial bias built into a commonly used medical diagnostic algorithm for lung function may be leading to underdiagnoses of breathing problems in Black men. The study suggests that as many as 40% more Black male patients might have been accurately diagnosed if the software were not racially biased. The software algorithm adjusts diagnostic thresholds based on race, affecting medical treatments and interventions.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.3 Unequal performance across groups

The diagnostic software performs with unequal accuracy and effectiveness across demographic groups, specifically underdiagnosing Black patients due to race-based adjustments built into its design.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The underdiagnosis is directly caused by the outputs of the deployed diagnostic software, which unintentionally perpetuates historical racial biases while attempting to assess lung function.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The report describes an AI system used in healthcare as a diagnostic tool, which is explicitly classified as High Risk under the EU AI Act due to its significant implications for patient safety and fundamental rights.

AI system and alleged parties

• AI system: None named
• AI purpose: Medical Diagnosis Support; Value Estimation
• Behaviour type: Assistant
• Alleged developer: unknown
• Alleged deployer: University of Pennsylvania Health System
• Alleged harmed parties: Black men who underwent lung function tests between 2010 and 2020 and potentially received inaccurate or delayed diagnoses and medical interventions due to the biased algorithm

Harm severity

Highest direct severity in any category: Severe. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Substantial
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Substantial
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Physical

Reported: Yes, the report describes how the software's race-based adjustments may prevent Black patients from getting timely medications, medical procedures, or lung transplants.

Directly caused: N/A

Indirectly caused: Nearly 400 Black men in the study were underdiagnosed with lung obstructions or impairments, which indirectly delayed or prevented necessary medical treatments and interventions.

Inferred additional harm: It is highly likely that thousands of Black patients nationwide have suffered worsened respiratory health, advanced lung disease, or preventable death due to delayed care caused by this widespread diagnostic bias.

Differential treatment

Reported: Yes, the report explicitly describes how the software raises the diagnostic threshold specifically for Black patients, leading to fewer diagnoses compared to white patients.

Directly caused: The software applied race-based adjustments that treated Black patients differently by requiring a lower level of measured lung function to trigger a diagnosis.

Indirectly caused: Black patients were less likely to be referred for specialized care, prescribed medications, or placed on lung transplant lists compared to white patients with similar actual lung function.

Inferred additional harm: Systemic racial disparities in healthcare access and clinical outcomes were reinforced across the medical system where these algorithms are deployed.

People affected

• Occurrences reported: 1
• People reportedly harmed: 400
• People reportedly exposed: 2700

Potential causes

Management

• Delayed software modernization: Hospital management fails to quickly deploy updated, unbiased algorithms.

Technology

• Race-adjusting software design: Software raises diagnostic thresholds for Black patients based on race.
• Fragmented system software: Hospitals use varied software versions, delaying system-wide updates.

Data Inputs

• Biased baseline measurements: Historical reference data adjusted scores based on patient race.

Human Factors

• Historical clinical bias: Clinicians historically assumed Black lungs were innately inferior.
• Automation bias in diagnosis: Clinicians rely on software scores instead of clinical symptoms.

Process and Methods

• Biased diagnostic guidelines: Medical standards institutionalized race-based adjustments in software.

Regulatory Environment

• Lack of algorithmic standards: No regulatory body mandated the removal of race-based software adjustments.

Information quality

• Classification confidence: High
• Reason for confidence: The report is highly detailed, citing a specific peer-reviewed study in 'JAMA Network Open' with clear statistics (e.g., 40% more diagnoses, nearly 400 additional cases, sample sizes of 2,700 Black men and 5,700 white men). The role of the software and the nature of the bias are explicitly described.
• Ambiguities identified: The specific developers or brand names of the spirometry software are not identified.
• Alternative interpretations: None. The findings of the study regarding algorithmic bias are clear and uncontested in the text.

The incident involves embedded racial bias in medical diagnostic software that underdiagnoses respiratory conditions in Black patients. While it poses a serious concern for healthcare equity and civil rights, its direct national security implications are minor and restricted to domestic public health software systems.

• Overall national security impact: Minor
• Response level: Moderate
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The issue represents an ongoing, systemic clinical software bias rather than an active crisis requiring immediate national security intervention.
• Autonomy: Human-supervised. The diagnostic software acts as a clinical decision-support tool, providing assessments that require final review and action by medical professionals.
• Novelty: Established threat. Algorithmic bias and race-based adjustments in medical software are well-documented, existing issues within the healthcare sector.

Impact by dimension

• Physical security: Negligible. The incident involves clinical diagnostic software and does not present threats to physical systems, critical infrastructure, kinetic targeting, or military operations.
• Information security: Negligible. There is no indication of information warfare, intelligence compromise, espionage, or malicious data theft associated with this medical software.
• Sovereignty: Negligible. The incident does not impact state authority, territorial control, border security, or core government administrative decision-making.
• Economic security: Negligible. The software bias does not threaten national financial systems, strategic technology supply chains, or overall economic security.
• Societal stability: Minor. The embedded algorithmic bias leads to systematic underdiagnosis and unequal healthcare outcomes for Black patients, representing a domestic civil rights and public health concern.