Meet the Secret Algorithm That's Keeping Students Out of College

In response to the Covid-19 pandemic, the International Baccalaureate final exams were replaced by a calculated score, prompting complaints of unfairness from teachers and students.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The primary failure was the statistical model's inability to robustly and accurately predict individual student performance, leading to erroneous and volatile grade assignments.

Additional risk subdomains

• 7.4 Lack of transparency or interpretability: The IB refused to disclose the formula or methodology of the grading algorithm, preventing external scrutiny and leaving students unable to understand how their grades were determined.
• 1.1 Unfair discrimination and misrepresentation: The algorithm unfairly represented students' individual capabilities by tying their grades to historical school performance and pooling data from other schools, resulting in unequal treatment.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The academic and financial harms were directly caused by the inaccurate grade predictions generated by the deployed IB statistical model, which was intended to provide fair grades but unintentionally produced flawed outcomes.

EU AI Act risk tier

• Risk tier: 2 High Risk

Risk Level 2: High Risk. The system is an educational assessment tool that directly determines student grades and university admissions, which falls under 'Educational and vocational training systems affecting access to education and employment'.

AI system and alleged parties

• AI system: IB grading algorithm
• AI purpose: Automatic Skill Assessment; Value Estimation
• Behaviour type: Tool
• Alleged developer: International Baccalaurette
• Alleged deployer: International Baccalaurette
• Alleged harmed parties: International Baccalaureate students

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Minor, indirect Minor
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Minor, indirect Minor
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: The report explicitly describes financial losses, including students losing university scholarships and facing fees of about EUR 700 (USD 791) to retake exams in November.

Directly caused: Students who chose to retake the exams in person to correct their grades faced direct costs of approximately EUR 700 ($791) per student.

Indirectly caused: Several students lost critical university scholarships to top universities, which represents substantial indirect financial harm.

Inferred additional harm: With over 15,000 petition signers and many losing scholarships or paying appeal fees, the collective financial loss across the global student body likely reached millions of dollars.

Differential treatment

Reported: The report explicitly describes differential treatment, noting that students' grades were calculated differently depending on their school's historical data or class size.

Directly caused: The algorithm used historical school performance and pooled data from other schools, meaning students at schools with less historical data or smaller classes were subjected to noisier and potentially lower predictions.

Inferred additional harm: Students from historically underperforming or smaller schools were likely systematically disadvantaged compared to peers at larger, historically high-performing schools, reinforcing systemic educational inequalities.

Civil rights

Reported: The report explicitly describes concerns regarding the fairness of using a formula to determine grades that shape a person's opportunities in life, which touches on the right to education and fair assessment.

Directly caused: The deployment of an opaque, flawed algorithm directly undermined students' rights to fair academic assessment and equal access to higher education.

Inferred additional harm: Thousands of students globally may have had their educational and career trajectories unfairly altered without due process or transparent recourse.

Psychological

Reported: The report explicitly describes psychological distress, with students reporting being 'extremely shocked' and teachers being 'floored' by the unexpected results.

Directly caused: Students like Anahita Nagpal experienced severe shock and distress upon finding their medical career plans derailed by the algorithm's output.

Indirectly caused: Parents and teachers experienced significant anxiety and frustration trying to support students who lost university places and scholarships.

Inferred additional harm: Given that thousands of students received unexpectedly low grades, it is highly likely that a large portion of the 170,000 students experienced elevated anxiety, stress, and grief over disrupted life plans.

People affected

• Occurrences reported: 1
• People reportedly harmed: 15000
• People reportedly exposed: 170000

Potential causes

Management

• Inadequate Risk Assessment: Management failed to assess the risks of unfair automated grading.
• Poor Accountability: Management declined to explain the model and charged for appeals.

Technology

• Flawed Grading Algorithm: The statistical formula predicted lower than expected scores for students.
• Bespoke Equation Mismatch: Bespoke equations for each school failed to accurately predict individual work.

Data Inputs

• Historical School Data Use: Basing individual grades on past school performance created unfairness.
• Data Pooling for New Schools: Using pooled data from other schools led to inaccurate grade predictions.
• Noisy Small Class Data: Smaller classes had fewer data points, generating noisier results.

Human Factors

• Unequal Student Effort: Late communication meant some students put less effort into assignments.

Process and Methods

• Lack of Model Transparency: The formula and inputs were not disclosed for external scrutiny.
• Hastily Deployed System: The system was rapidly deployed during Covid-19 without proper testing.
• Inconsistent Deadlines: Assignments were submitted before students knew they would determine grades.

Regulatory Environment

• Lack of Algorithmic Regulation: No regulatory standards governed the use of algorithms for grading.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides clear, detailed factual accounts of the algorithm's deployment, its inputs, and the direct consequences on students' lives. The role of the AI system is explicitly stated, and the scale of the impact is well-documented.
• Ambiguities identified: The exact mathematical formula and specific weightings of the inputs (assignments vs. historical school data) are not disclosed by the IB.
• Alternative interpretations: None, as the causal link between the algorithm's output and the resulting academic/financial harm is clear and uncontested.

The International Baccalaureate's deployment of a flawed grading algorithm in 2020 caused widespread disruption to university admissions and scholarships globally. While causing notable societal frustration and individual financial harm, the incident holds negligible direct national security implications.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Global
• Alleged perpetrator: International Baccalaureate

Threat characteristics

• Imminence: Long-term. The incident represents a policy and algorithmic governance concern rather than an active national security crisis.
• Autonomy: Human-supervised. The system automatically calculated grades based on predefined inputs, with the IB organization maintaining overall policy control and a manual appeals process.
• Novelty: Evolved capability. Represents an advancement in the scale of using statistical models for high-stakes automated decision-making in public education.

Impact by dimension

• Physical security: Negligible. The incident involved an educational grading algorithm and posed no threat to physical systems, critical infrastructure, or human safety.
• Information security: Negligible. No intelligence compromise, classified data theft, or information warfare operations were associated with this incident.
• Sovereignty: Negligible. The incident affected a private educational foundation's grading system and did not compromise state authority or core government operations.
• Economic security: Negligible. While individual students faced financial losses regarding scholarships and appeal fees, there was no threat to strategic industries or national economic security.
• Societal stability: Minor. The grading algorithm caused widespread public distress, protests, and potential inequality in educational access, representing a minor, localized societal impact.