Opera's GPT-Based AI Reportedly Accused War Photographers of War Crimes

When prompted about "photographers accused of committing war crimes," Opera's GPT-based chatbot Aria provided a list of photographers who take photography of military conflicts.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The AI system generated false and defamatory accusations against real individuals, leading to the spread of incorrect and harmful information.

Additional risk subdomains

• 7.3 Lack of capability or robustness: The chatbot failed to perform reliably by hallucinating severe false accusations due to a lack of fact-checking capabilities.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The defamatory output was caused by the AI chatbot generating unintended hallucinations after being deployed to the public.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system is an 'AI chatbot called Aria' which falls under Limited Risk due to transparency obligations for user-interacting AI systems.

AI system and alleged parties

• AI system: Aria (Opera)
• AI purpose: Chatbot; Question Answering
• Behaviour type: Assistant
• Alleged developer: Opera, OpenAI
• Alleged deployer: Opera
• Alleged harmed parties: Ronald L. Haeberle, Ron Haviv, Raymond D’Addario, Lynsey Addario, Lee Miller, Larry Towell, James Nachtwey

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes toxic and defamatory content generated directly by the AI system.

Directly caused: The chatbot generated defamatory statements falsely accusing respected photographers of war crimes.

Psychological

Reported: The report explicitly describes reputational harm which can cause psychological distress to the falsely accused individuals.

Directly caused: The chatbot slandered acclaimed photographers by falsely accusing them of committing war crimes, potentially causing severe distress and reputational damage to the 5 named individuals.

Epistemic

Reported: The report explicitly describes epistemic harm caused directly and indirectly by the incident.

Directly caused: The chatbot fabricated false accusations of war crimes, presenting hallucinations as facts to the user.

Indirectly caused: The author notes that such AI chatbots 'undermine our relationship with knowledge and drive us towards an epistemological crisis' by flooding the information space with unverified content.

People affected

• Occurrences reported: 1
• People reportedly harmed: 5
• People reportedly exposed: 5

Potential causes

Management

• Premature Product Release: Management released the chatbot browser integration without safeguards.

Technology

• Statistical Word Associations: AI generates text based on shifting probabilities of words.
• Lack of Fact-Checking: The chatbot completely lacks the ability to fact check its outputs.
• Blackbox AI Correlations: AI operates as a blackbox of obscure correlations between words.

Data Inputs

• Live Web Results Integration: Integrating live web results without verifying their accuracy.

Process and Methods

• Lack of Rigorous Testing: The AI product was released to the public without adequate testing.
• No Transparent Editorial Policy: System lacks transparent editorial policies to guide generated content.
• Lack of Ethical Frameworks: The chatbot outputs are not constrained by common sense or ethics.

Regulatory Environment

• Lack of Regulatory Oversight: No regulatory framework prevents dumping untested AI on society.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly details the specific AI system (Opera Aria based on GPT), the exact nature of the failure (hallucinating war crime accusations), and the specific individuals affected. There is little ambiguity about what occurred.

An integrated browser chatbot, Opera Aria, hallucinated and generated defamatory statements falsely accusing five prominent war photographers of committing war crimes. While the incident highlights ongoing challenges with LLM accuracy and epistemic security, its direct national security impact remains minor and localized to reputational harm for the affected individuals.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Norway, United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The incident represents an ongoing, long-term strategic challenge regarding large language model hallucinations rather than an active, imminent national security crisis.
• Autonomy: Human-supervised. The chatbot operates autonomously to generate text responses based on user queries, but remains within a consumer browser application context with human users evaluating the output.
• Novelty: Established threat. AI hallucinations and the generation of false or defamatory information by large language models are well-documented, established threats in the AI domain.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or kinetic targeting capabilities were affected or compromised in this incident.
• Information security: Minor. The AI chatbot generated false, defamatory accusations of war crimes against real individuals, demonstrating minor risks to information integrity and public trust in digital platforms.
• Sovereignty: Negligible. No sovereign government functions, electoral systems, or state decision-making processes were impacted by this commercial browser integration.
• Economic security: Negligible. No strategic technologies, financial systems, or critical supply chains were targeted or disrupted by the chatbot failure.
• Societal stability: Negligible. While the chatbot defamed five specific war photographers, the incident did not cause population-scale social manipulation, systemic discrimination, or civil unrest.