The Department of Justice's PATTERN recidivism risk assessment tool was found to produce racially biased results, leading to the misclassification of thousands of inmates. The algorithm disproportionately overpredicted the risk for Black, Hispanic, and Asian inmates, which restricted their access to early-release programs under the First Step Act. The Justice Department has acknowledged these disparities and is currently working on revisions to the tool.
Department of Justice’s inmate-recidivism risk assessment tool was reported to have produced racially uneven results, misclassifying risk levels for inmates of color.
Risk classification
- Primary risk domain: 1 Discrimination & Toxicity
- Primary risk subdomain: 1.3 Unequal performance across groups
The PATTERN algorithm demonstrated unequal accuracy and performance across demographic groups, overpredicting recidivism risk for Black, Hispanic, and Asian inmates compared to white inmates.
Additional risk subdomains
- 7.3 Lack of capability or robustness: The system suffered from fundamental math and human errors that led to the misclassification of 14,000 individuals.
Causal factors
- Entity: AI
- Intent: Unintentional
- Timing: Post-deployment
The racial disparities and misclassifications were caused by the PATTERN AI system's biased risk predictions after it was deployed in the federal prison system.
EU AI Act risk tier
High Risk: The PATTERN algorithm is used in law enforcement and the administration of justice to determine eligibility for early prison release, which directly impacts individual liberty and fundamental rights.
AI system and alleged parties
- AI system: Pattern
- AI purpose: Recidivism Prediction; Judicial Decision Support
- Behaviour type: Tool
- Alleged developer: US Department of Justice
- Alleged deployer: US Department of Justice
- Alleged harmed parties: inmates of color
Harm severity
Severity ratings are based on this scale.
Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.
- Physical: direct Negligible, indirect Negligible
- Infrastructure: direct Negligible, indirect Negligible
- Property: direct Negligible, indirect Negligible
- Financial: direct Negligible, indirect Negligible
- Environmental: direct Negligible, indirect Negligible
- Malicious content: direct Negligible, indirect Negligible
- Differential treatment: direct Substantial, indirect Negligible
- Civil rights: direct Substantial, indirect Negligible
- Democracy: direct Negligible, indirect Negligible
- Privacy: direct Negligible, indirect Negligible
- Psychological: direct Negligible, indirect Negligible
- Epistemic: direct Negligible, indirect Negligible
- Child sexual exploitation and abuse: direct Negligible, indirect Negligible
Differential treatment
Reported: Yes, the report explicitly describes systemic differential treatment based on race.
Directly caused: The PATTERN algorithm overpredicted the recidivism risk for Black, Hispanic, and Asian inmates, resulting in only 7% of Black people being classified as minimum risk compared to 21% of white people.
Inferred additional harm: The racial disparities in the algorithm's predictions likely reinforced systemic inequalities within the federal prison system, disproportionately keeping people of color incarcerated longer.
Civil rights
Reported: Yes, the report explicitly mentions concerns raised by civil rights groups regarding racial disparities and liberty.
Directly caused: The misclassification of 14,000 inmates in the wrong risk categories directly impacted their liberty by denying them access to early-release programs they were legally entitled to under the First Step Act.
Inferred additional harm: The systemic denial of early release credits based on a biased algorithm represents a broader violation of civil rights and equal protection under the law for thousands of incarcerated individuals.
People affected
- Occurrences reported: 1
- People reportedly harmed: 14000
- People reportedly exposed: 14000
Potential causes
Management
- Rushed Implementation Decisions: Management pushed the tool live despite known flaws and racial disparities.
- Lack of Transparency: Management failed to provide open auditing or transparent risk assessment.
Technology
- Mathematical and Coding Errors: The algorithm suffered from math errors that misclassified 14,000 inmates.
- Algorithmic Overprediction: Pattern overpredicted recidivism risk for Black, Hispanic, and Asian inmates.
- Algorithmic Underprediction: Pattern underpredicted violent recidivism risk for some inmates of color.
Data Inputs
- Biased Criminal History Data: Criminal history inputs reflect systemic overpolicing of minority communities.
- Socioeconomic Proxy Factors: Education and restitution factors correlate heavily with race and class.
Human Factors
- Human Programming Errors: Developers made manual and logical errors during the implementation phase.
- False Aura of Objectivity: Users trusted the tool's 'patina of science' over critical human judgment.
Process and Methods
- Rushed Tool Development: The Justice Department rushed the tool's creation to meet tight deadlines.
- Inadequate Testing and Validation: The initial roll-out lacked rigorous validation to detect disparate impact.
Regulatory Environment
- Strict Congressional Deadlines: Congress imposed tight statutory deadlines under the First Step Act.
- Lack of Auditing Standards: No formal regulatory oversight was initially required for algorithmic fairness.
Information quality
- Classification confidence: High
- Reason for confidence: The report provides clear, authoritative details from the Justice Department's own findings, including the specific number of affected individuals (14,000) and the nature of the racial disparities. There is high consensus among the sources cited, and the role of the PATTERN algorithm is explicitly defined.
- Ambiguities identified: The exact mathematical errors and the specific weights of the policy decisions/factors in the algorithm are not fully detailed in the text.
- Alternative interpretations: None. The facts of the algorithmic bias and its consequences are uncontested by the Justice Department.
The Department of Justice's PATTERN risk assessment tool produced racially biased results, misclassifying approximately 14,000 inmates and restricting eligible inmates of color from early-release programs. While causing notable domestic civil rights concerns, the incident has minimal direct implications for broader national security.
- Overall national security impact: Minor
- Scope: Single nation
- Primary target: United States
- Alleged perpetrator: Unknown
Threat characteristics
- Imminence: Long-term. Represents an ongoing policy and systemic challenge rather than an active, fast-moving crisis.
- Autonomy: Human-supervised. The algorithm served as a decision-support tool for corrections officials, who maintained final authority over inmate release.
- Novelty: Established threat. Algorithmic bias in risk assessment and criminal justice tools is a well-documented and previously observed issue.
Impact by dimension
- Physical security: Negligible. No physical systems, critical infrastructure, or kinetic targeting capabilities were involved or affected in this incident.
- Information security: Negligible. The incident did not involve intelligence compromise, espionage, or information warfare operations.
- Sovereignty: Minor. Algorithmic errors temporarily compromised federal prison release decision-making processes, but without threatening state authority or territorial control.
- Economic security: Negligible. The incident did not affect strategic technologies, critical supply chains, or financial system stability.
- Societal stability: Minor. Systemic discrimination and civil rights impacts occurred due to the misclassification of 14,000 inmates, but it did not lead to widespread civil unrest.