The Risks of Machine Learning Systems

What are the risks from AI?

‍

This week we spotlight the fifteenth risk framework included in the AI Risk Repository: 

‍

Tan, S., Taeihagh, A., & Baxter, K. (2022). The Risks of Machine Learning Systems. In arXiv [cs.CY]. arXiv. http://arxiv.org/abs/2204.09852

‍

‍

This paper presents the Machine Learning System Risk framework (MLSR). This framework categorises the risks of ML systems into first-order and second-order risks, and the factors that contribute to them. 

‍

First-order risks stem from aspects of the ML system, and the choices made during its conception, design and implementation. Second-order risks stem from the consequences of first-order risks (e.g., system failures that result from design and development choices), and occur when an ML system interacts with the world. 

‍

MLSR first-order risk categories include: 

• Application 
• Algorithm 
• Robustness 
• Misapplication 
• Design 
• Control 
• Train/Val. Data 
• Implementation 
• Emergent behaviour 

‍

MLSR second-order risk categories include: 

• Safety 
• Privacy 
• Environmental 
• Discrimination
• Security 
• Organizational 

‍

Key features of the framework and associated paper:

• Explores how different risks may manifest in various types of ML systems, the factors that affect each risk, and how first-order risks may lead to second-order effects when the system interacts with the real world.
• Shows how real events and prior research can be categorised according to their framework. 
• Offers a holistic way in which to conduct risk assessments of ML systems. 
• Constructs a comprehensive taxonomy by integrating multiple sources including algorithmic impact assessments, software risks, ML literature surveys, incident reports, and professional experience.

Disclaimer‍

This summary highlights a paper included in the MIT AI Risk Repository. We did not author the paper and credit goes to Samson Tan, Araz Taeihagh and Kathy Baxter. For the full details, please refer to the original publication: https://doi.org/10.48550/arXiv.2204.09852. 

Further engagement 

→ View all the frameworks included in the AI Risk Repository 

→ Sign-up for our project Newsletter 

‍