Algorithm Used by the Malaysian Judiciary Reportedly Recommended Unusually High Sentencing to a Drug Possession Case

The AI system used by the Malaysian judiciary which explicitly considered age, employment, and socio-economic data provided sentencing to a drug possession case that was alleged by lawyer to be disproportionately high for the crime committed.

Risk classification

• Primary risk domain: 5 Human-Computer Interaction
• Primary risk subdomain: 5.1 Overreliance and unsafe use

The primary risk identified is the potential for judges to overrely on machine-generated recommendations, compromising independent judicial discretion.

Additional risk subdomains

• 7.4 Lack of transparency or interpretability: The AI tool is described as an 'opaque algorithm' with little transparency regarding how its parameters are weighted.
• 1.1 Unfair discrimination and misrepresentation: Critics warned that the AI risks entrenching and amplifying bias against minorities and marginalized groups due to its training data.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The risk of unconstitutional sentencing and overreliance on algorithmic recommendations is driven by the human decision to deploy and use the tool in court.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The AI system is used in the administration of justice and democratic processes (specifically, judicial sentencing), which has significant implications for fundamental rights and liberty. The reports describe it as an AI tool used by the judiciary to recommend prison sentences.

AI system and alleged parties

• AI system: AI sentencing tool (Sarawak Information Systems)
• AI purpose: Judicial Decision Support
• Behaviour type: Assistant
• Alleged developer: Sarawak Information Systems
• Alleged deployer: Malaysian judiciary, Malaysian courts
• Alleged harmed parties: Malaysian convicted people

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 1
• People reportedly harmed: 2
• People reportedly exposed: 2

Potential causes

Management

• Prioritizing Speed Over Safety: The system was deployed to clear backlogs quickly without addressing risks.
• Inadequate Risk Assessment: Management failed to fully resolve issues of algorithmic bias before deployment.

Technology

• Opaque Algorithm: The AI tool operates as a black box without transparent decision logic.
• Inability to Exercise Discretion: The AI lacks a human judge's ability to weigh individual mitigating factors.
• Risk of Bias Amplification: The system risks amplifying historical biases in sentencing.

Data Inputs

• Limited Training Dataset: The algorithm was trained on only five years of data from 2014 to 2019.
• Omission of Case Nuances: Structured parameters fail to capture complex qualitative mitigating factors.

Human Factors

• Cognitive Bias and Automation Bias: Young magistrates may accept the AI's recommendations without question.
• Lack of System Understanding: Lawyers, judges, and the public do not fully understand how the AI works.

Process and Methods

• No Guidelines for Practitioners: The Bar Council was not given guidelines or opportunities to provide feedback.
• Inadequate Consultation Process: The pilot was launched without proper consultation with key legal stakeholders.

Regulatory Environment

• No Provision in Criminal Code: The Criminal Procedure Code does not provide for the use of AI in courts.
• Constitutional Challenges: The tool's use potentially violates Articles 5(1) and 8(1) of the Constitution.

Information quality

• Classification confidence: High
• Reason for confidence: The reports are detailed, consistent, and cover the legal objections, the specific cases, and the broader context of the pilot.
• Ambiguities identified: The exact formula or weightings of the parameters (age, employment, socio-economic data) used by the AI are not fully detailed.
• Alternative interpretations: The incident could be interpreted purely as a constitutional/legal dispute over judicial procedure rather than an AI safety failure, though the risk of algorithmic bias and overreliance makes it an AI safety issue.

The Malaysian judiciary piloted an AI sentencing tool, raising constitutional and human rights concerns regarding due process, bias, and the erosion of judicial discretion. While it directly touches on core government functions, the national security threat remains minor and manageable within standard legal and constitutional frameworks.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: Malaysia
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The deployment represents a long-term strategic and legal policy concern rather than an immediate crisis requiring urgent national security response.
• Autonomy: Human-supervised. The AI system provides sentencing recommendations, but the human magistrate retains final decision-making authority and can deviate from the AI output.
• Novelty: Evolved capability. While AI sentencing tools have been used globally, implementing them in Malaysian courts represents an evolved capability and a new local legal precedent.

Impact by dimension

• Physical security: Negligible. The incident involves a judicial sentencing recommendation tool and does not impact physical systems, critical infrastructure, or human safety.
• Information security: Negligible. No evidence of information warfare, intelligence compromise, espionage, or systematic disinformation campaigns targeting national security.
• Sovereignty: Minor. The pilot directly impacts judicial decision-making, a core government function. However, the threat is minor as it is a domestic legal pilot subject to constitutional challenges and standard judicial oversight.
• Economic security: Negligible. The incident does not pose threats to financial stability, strategic industries, or technological competitive advantage.
• Societal stability: Minor. The tool raises concerns about constitutional rights to liberty and equal protection, with risks of algorithmic bias. However, the impact is currently limited to a localized judicial pilot.