AI Incident Tracker

Harm Taxonomy

This scale rates the severity of harm from each incident in the AI Incident Database, using an LLM pipeline built and run by Arcola AI to assess reports of the incident. It has 13 harm categories, each with five levels from Negligible to Catastrophic. The original ten categories are based on the CSET AI Harm Taxonomy; Psychological Harm, Epistemic Harm and Child Sexual Exploitation and Abuse were added for this tracker. The scale rates individual incidents, so it does not capture diffuse harms that build up across many events, such as labor displacement. An incident can cause several types of harm, so counts overlap across categories, and Negligible includes incidents with no reported harm of that type.

‍

‍

Important context for interpreting these results:

The data presented is the output of an LLM classifier pipeline applied to the raw reports from the AI Incident Database (AIID), which relies on submissions from the public and subject matter experts. The quality, reliability and depth of detail in the reports varies across the dataset. As the reporting is voluntary, the dataset is inevitably subject to some degree of sampling bias.
The LLM classification tool has been developed iteratively, and its agreement with human expert consensus is comparable to the agreement between 2 independent human experts (article forthcoming). Spot-checks have been used to provide feedback on misclassifications and to iterate the tool, improving its reliability, however there are likely to remain incidents where the LLM classification does not match expert consensus.

Therefore patterns and trends observed in the data should be taken as indicative and validated through further analysis.

‍

Explore Other Views