Companies × risks
Exposure to toxic content
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
10 of the world's largest companies we reviewed acknowledge the AI risk "Toxic content", including Microsoft, Salesforce and IBM.
Which companies acknowledge this risk?
- Microsoft: “The Copilot experience can potentially produce other types of inappropriate or offensive content... AI-generated images that potentially contain harmful artifacts such as hate symbols... and sexually charged content that evades sexual-related content filters.” (Transparency Note for Microsoft Copilot)
- Salesforce: “scan and categorize generated content in real time for signs of hate speech... harassment... Should harmful content be identified, the system's content filtering will automatically filter, block, or flag the output” (Trusted AI: Key Principles | Salesforce)
- IBM: “Toxic output occurs when the model produces hateful, abusive, and profane (HAP) or obscene content... can adversely impact and harm people interacting with the model.” (Foundation models: Opportunities, risks and mitigations)
- Meta: “we evaluated each fine-tuned model against a range of risks, including in areas like violent crime, child sexual exploitation” (Connect 2024: The Responsible Approach We're Taking to Generative AI)
- Alphabet (Google): “Our policies for mitigating harm in areas such as child safety, suicide, and self-harm ... guide our models and products to minimize certain types of harmful outputs.” (Google - OECD HAIP Report)
- SAP: “prevent the AI from making... inappropriate, or offensive suggestions... and block any inappropriate ones before they reach the user.” (SAP AI Ethics Handbook 2025)
- Amazon: “prohibits the use... (4) for harm or abuse of a minor, including grooming and child sexual exploitation” (Amazon 2025 Proxy Statement)
- Apple: “The introduction of new and complex technologies, such as artificial intelligence features, can increase these and other safety risks, including exposing users to harmful... content and experiences.” (Apple 2025 Annual Report on Form 10-K)
- Baidu: “Increased risk of users' rights being compromised due to the dissemination of harmful content such as implicit bias, violence, terrorism, etc. ... We have developed targeted content governance measures to address harmful AI-generated content.” (Baidu 2025 Environmental, Social and Governance Report)
- AppLovin: “AI technologies, including generative AI, may create content that is, or is perceived to be... offensive, unethical” (AppLovin Corporation 10K - FY24)