Kidney Testing Method Allegedly Underestimated Risk of Black Patients

Decades-long use of the estimated glomerular filtration rate (eGFR) method to test kidney function which considers race has been criticized by physicians and medical students for its racist history and inaccuracy against Black patients.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.3 Unequal performance across groups

The algorithm performs with unequal accuracy across demographic groups, systematically overestimating kidney health for Black patients compared to white patients, leading to unequal medical outcomes.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Doctors relied on the algorithm's race-based recommendations without questioning them, leading to zero of the 64 qualifying Black patients being referred for transplants.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The clinical harm is caused by the automated output of the deployed eGFR algorithm, which unintentionally underestimated kidney disease severity in Black patients.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The system is an AI tool used in healthcare for clinical decision-making and diagnostic support, which has significant implications for safety and fundamental rights.

AI system and alleged parties

• AI system: eGFR kidney function algorithm
• AI purpose: Medical Diagnosis Support; Resource Allocation
• Behaviour type: Tool
• Alleged developer: Chronic Kidney Disease Epidemiology Collaboration
• Alleged deployer: Chronic Kidney Disease Epidemiology Collaboration
• Alleged harmed parties: Black patients, African-American patients

Harm severity

Highest direct severity in any category: Severe. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Substantial
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Substantial
• Civil rights: direct Minor, indirect Substantial
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Physical

Reported: The report explicitly describes indirect physical harm through delayed medical care.

Directly caused: N/A

Indirectly caused: 64 Black patients who qualified for the kidney transplant waitlist were not referred, evaluated, or waitlisted, delaying critical life-saving treatment.

Inferred additional harm: It is highly likely that some of the 64 patients, and thousands of others nationwide, suffered severe physical deterioration or death due to delayed kidney transplants.

Differential treatment

Reported: The report explicitly describes systematic differential treatment of Black patients.

Directly caused: The algorithm automatically boosted Black patients' eGFR scores by 15.9 percent, treating them differently based solely on race.

Indirectly caused: Black patients received lower-quality care, delayed specialist referrals, and delayed transplant evaluations compared to non-Black patients.

Inferred additional harm: Systemic medical discrimination affecting thousands of Black kidney disease patients across the United States.

Civil rights

Reported: The report explicitly describes violations of civil rights and equal access to healthcare.

Directly caused: The algorithm's design systematically discriminated against Black patients, violating their right to equal treatment.

Indirectly caused: Black patients were denied equal access to life-saving medical resources like kidney transplants.

Inferred additional harm: Widespread civil rights violations in healthcare delivery across multiple medical institutions nationwide.

People affected

• Occurrences reported: 1
• People reportedly harmed: 743
• People reportedly exposed: 2225

Potential causes

Management

• Institutional Inertia: Hospitals delayed changing formula until prompted by student petitions.
• Delayed Policy Updates: Management waited for society guidelines before updating local formulas.

Technology

• Flawed Race-Correction Formula: Formula boosts Black patient scores by 15.9 percent, masking kidney disease.
• Lack of Physiological Basis: Equation incorrectly equates social category of race with biological difference.

Data Inputs

• Small Black Patient Sample: Original 2009 study used small sample size of Black patients to build formula.
• Binary Categorization of Race: Formula simplistically groups patients into only Black or non-Black.
• Confounding Creatinine Factors: Formula ignored non-racial factors affecting creatinine like diet.

Human Factors

• Uncritical Trust in Algorithm: Doctors accepted race-based scores without questioning clinical validity.
• Education in Biased Science: Medical training historically taught race as a biological variable.

Process and Methods

• Outdated Medical Guidelines: Societies maintained guidelines endorsing the race-adjusted formula.
• Lack of Care Plan Reviews: Hospitals failed to review historical patient care plans for racial bias.

Regulatory Environment

• Delayed Federal Investigation: HHS and CMS did not investigate race-adjusted algorithms for years.
• Lack of Algorithmic Standards: No regulatory standards existed to prevent bias in clinical calculators.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, peer-reviewed scientific study results detailing the exact number of patients affected within a specific healthcare system, along with the precise mathematical adjustment used.

A widely used clinical algorithm for assessing kidney function incorporated a race-based multiplier that systematically underestimated the severity of kidney disease in Black patients, delaying critical treatments like transplants. While presenting significant ethical and civil rights concerns within the healthcare sector, the incident has minor direct national security implications and is being addressed through institutional policy reforms.

• Overall national security impact: Minor
• Response level: Moderate
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. This represents a chronic, systemic bias issue within medical software rather than an active, imminent national security crisis.
• Autonomy: Human-controlled. The eGFR formula is a static mathematical tool that assists clinicians, who ultimately retain the authority to make final medical and referral decisions.
• Novelty: Established threat. Algorithmic bias and the use of race-based multipliers in clinical calculations are well-documented historical issues in healthcare.

Impact by dimension

• Physical security: Negligible. The incident involves clinical algorithms and delayed medical care, which does not present a direct threat to physical critical infrastructure or national safety systems.
• Information security: Negligible. No information warfare, intelligence compromise, or systematic foreign disinformation campaigns are associated with this incident.
• Sovereignty: Negligible. The issue is limited to clinical decision-making and does not threaten state authority, territorial control, or core government operations.
• Economic security: Negligible. There are no indications of strategic technology theft, financial system attacks, or direct threats to economic security.
• Societal stability: Minor. The algorithm systematically discriminated against Black patients, leading to unequal healthcare outcomes. However, the impact is manageable through standard institutional and medical policy reforms.