Companies × risks
Lack of capability or robustness
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
20 of the world's largest companies we reviewed acknowledge the AI risk "Capability & robustness", including Microsoft, IBM and Salesforce.
Which companies acknowledge this risk?
- Microsoft: “many existing widely used benchmarks are not robust to distribution shifts and other measures of validity over time and fail to translate well to real-world settings” (Microsoft: G7 Hiroshima AI Process (HAIP) Transparency Report)
- IBM: “Poor model accuracy occurs when a model's performance is not sufficient to the task it was designed for... Low accuracy may occur if the model is not correctly engineered, or there are changes to the model's expected inputs.” (Foundation models: Opportunities, risks and mitigations)
- Salesforce: “These measures work to ensure that AI agents operate only within their intended domains, minimizing errors and reducing the likelihood of "hallucinations" or incorrect outputs.” (Salesforce Trusted AI Impact Report)
- SAP: “There are two major technical risks intrinsic to GenAI... lack of determinism. If these risks are left unmitigated, they will severely impede forensic analysis and security testing efforts.” (SAP Generative AI Cybersecurity Strategy)
- Palantir: “AI systems are already in wide, and rapidly growing, use... and they can and do fail in many different ways... leading to real harms... provisions to address the inherent brittleness of AI technologies... the tendency of models be anti-resilient to changes in data, environment, application context, etc.” (Palantir Australia Response to Supporting Responsible AI ...)
- Meta: “it is possible that the actual numbers of unique people using our products may vary significantly from our estimates, potentially beyond our estimated error margins… our data limitations may affect our understanding of certain details of our business and increase the risk of error for our Family metrics estimates.” (Meta - Form 10-K 2025 Annual Report)
- ServiceNow: “Limitations: Highlights any known weaknesses or situations where the model might not perform well.” (Understanding our Model Cards and Their Role in Responsible AI | ServiceNow)
- Amazon: “detect hallucinations in model responses that are not grounded in enterprise data or are irrelevant to the users' queries” (Amazon 2025 Proxy Statement)
- Nomura Research Institute: “AI learning from unintended data and exhibiting unpredictable behavior” (NRI - ESG Databook 2025)
- Visa: “as agentic commerce solutions scale, we may see increased instances of erroneous or disputed payments, increased chargebacks and reputational harm.” (Visa 2025 Annual report)
- Alphabet (Google): “disclaimers to set clear expectations - such as reminding people that the AI-generated outputs may contain inaccuracies and that they should take steps to verify information generated by the tool.” (Responsible AI Progress Report (February 2025))
- Citigroup: “ineffective, inadequate or faulty Generative AI development or deployment practices... could result in unintended consequences, such as AI algorithms that produce inaccurate or incomplete output” (Citigroup 2024 Annual report)
- Novartis: “AI algorithms may be flawed, insufficient, of poor quality... or contain other errors or inadequacies - any of which may not be easily detectable. AI systems are susceptible to producing false or "hallucinatory" inferences or outputs” (Novartis Annual Report 2025 - Form 20-F)
- American Express: “we increasingly use models that leverage artificial intelligence, which are subject to additional risks such as... inaccurate results” (American Express 2024 Annual Report)
- Eli Lilly: “AI-generated content, analyses, or recommendations we utilize could be deficient... there are technical challenges associated with achieving desired levels of accuracy, efficiency, and reliability. The algorithms and models utilized in AI systems may have limitations, including biases, errors, or inability to handle certain data types or scenarios” (Eli Lilly 2024 Annual Report)
- Johnson & Johnson: “We also acknowledge that the characteristics of data sets may change over time (called "drift"). This may require us to reexamine the data for new insights and evolve our models.” (Doing the Right Thing - AI & Ethics - Johnson & Johnson)
- Philip Morris International: “some information technology systems may be supported by artificial intelligence capabilities that may not function as intended” (Philip Morris International 2024 Annual Report)
- PNC Financial Services: “Enhanced use of automation may present its own risks. Automated systems may themselves experience outages or problems... Bad, missing or anomalous data can adversely affect the functioning of such tools.” (PNC Financial Services 2025 Annual Report)
- JPMorgan Chase: “The models and estimations that JPMorganChase uses, including those that use machine learning or artificial intelligence, may not be effective in all cases to identify, observe and mitigate risk due to a variety of factors” (JPMorganChase 2024 10-K Annual report)
- Walmart: “aimed at mitigating risks such as ... model robustness” (FY2025 ESG Report | Walmart Corporate)