Companies × risks
AI pursuing its own goals in conflict with human goals or values
AI systems acting in conflict with human goals or values, especially the goals of designers or users, or ethical standards. These misaligned behaviors may be introduced by humans during design and development, such as through reward hacking and goal misgeneralisation, or may result from AI using dangerous capabilities such as manipulation, deception, situational awareness to seek power, self-proliferate, or achieve other goals.
Alphabet (Google) and IBM are the only companies we reviewed that acknowledge the AI risk "AI misalignment".
Which companies acknowledge this risk?
- Alphabet (Google): “the risk of deceptive alignment, i.e. the risk of an AI system becoming aware that its goals do not align with human instructions, and deliberately trying to bypass the safety measures put in place by humans” (Taking a responsible path to AGI)
- IBM: “Alignment Misaligned Actions: AI agents can take actions that are not aligned with relevant human values, ethical considerations, guidelines, and policies.” (AI agents: Opportunities, risks, and mitigations — IBM AI Ethics Board)