%20(250%20x%20100%20px).png)
.png){kind=small}
AI incidents are on the rise, yet current databases struggle with inconsistent structure, limiting their utility for policymaking. The AI Incident Tracker project addresses this by creating a tool to classify AI incidents based on risks and harm severity. Using a Large Language Model (LLM), the tool processes raw reports from the AI Incident Database and categorizes them using established frameworks, such as the MIT Risk Repository and a harm severity rating system based on CSET’s AI Harm Taxonomy.
This project, led by Simon Mylius, provides a proof-of-concept analysis of reported AI incidents, including preliminary insights into trends in the available data.
You can explore the AI Incident Tracker using interactive visualizations, including data categorized using MIT Risk Taxonomies, changes in incidents over time, the severity of harm associated with incidents, and more.
The AI Incident Tracker is part of the MIT AI Risk Initiative, which aims to increase awareness and adoption of best practice AI risk management across the AI ecosystem.
This interactive visualization shows how incidents of harm from AI reported in the AI Incident Database are increasing over time, with the greatest increase in incidents associated with the Misinformation and Malicious Actors domains from the MIT AI Risk Repository. In the sections below, you can explore the incidents in more detail, for instance to see the growth in specific incidents related to particular risks or levels of harm.
You can explore different views of the database and classification in the project. For example, you can see all AI incidents classified using taxonomies from the MIT Risk Repository, the type of harm, and individual records in the AI Incident Database.
Key visualizations include bar charts and pie charts that display incident counts, proportions across domains (e.g., "System Failures," "Discrimination & Toxicity"), and trends in causal attributes. Additionally, insights highlight patterns such as the prevalence of system safety issues, intentional misuse trends, and incomplete reporting gaps.
Click through the links below to explore each of the interactive dashboards.
This classification database is intended to explore the potential capabilities and limitations of a scalable incident analysis framework. The classification analysis uses reports from the AI Incident Database (AIID) as input data which rely on submissions from the public and subject matter experts. The quality, reliability and depth of detail in the reports varies across the dataset. As the reporting is voluntary, the dataset is inevitably subject to some degree of sampling bias. Spot-checks have been used to provide feedback on misclassifications and to iterate the tool, improving its reliability, however a systematic validation study has not yet been completed.
Therefore patterns and trends observed in the data should be taken as indicative and validated through further analysis.
This work proves the concept of a scalable incident classification tool and paves the way for future work to explore its usefulness, validity, limitations and potential, which will be focused around the following activities:
The code repository will be made available as open-source to encourage users to evaluate and contribute improvements.
Please feel free to share feedback using this form - this will shape the direction of the work and help to make the tool as useful and relevant as possible.
