Purported AI-Generated War Footage Reportedly Circulated Widely Online During the Opening Phase of the War in Iran

Reports said that, in the early days of the war in Iran, purported AI-generated fakes showing nonexistent wartime scenes reached millions of viewers online. Their spread across social media and messaging apps allegedly distorted public perception of the conflict and contributed to information disorder.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale

The primary risk involves state-aligned and malicious actors using AI systems to conduct large-scale propaganda and disinformation campaigns during an active war.

Additional risk subdomains

• 3.2 Pollution of information ecosystem and loss of consensus reality: The massive volume of AI-generated fakes flooded social media feeds, creating an alternate reality and eroding the shared consensus of real-world events.
• 7.3 Lack of capability or robustness: AI systems like Grok failed to robustly enforce their own safety guardrails, allowing users to easily generate prohibited warzone imagery.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was driven by human actors intentionally generating and spreading AI-synthesized war propaganda to manipulate public opinion.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. The AI systems described are used to generate synthetic content (deepfakes, AI-generated images/videos), which under the EU AI Act are subject to transparency obligations to ensure users are informed that the content is AI-generated.

AI system and alleged parties

• AI system: Gemini, Grok, Sora (Google, OpenAI, xAI)
• AI purpose: Image Generation; Deepfake Video Generation
• Behaviour type: Tool
• Alleged developer: Unknown image generator developers, Unknown generative AI developers, Unknown deepfake technology developers
• Alleged deployer: Unknown social media accounts, Unknown disinformation actors
• Alleged harmed parties: social media users, People seeking information about the war in Iran, National security and intelligence stakeholders, General public, Epistemic integrity

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Severe
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Minor
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Severe
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the creation and spread of malicious propaganda and fake war imagery.

Directly caused: AI tools directly generated realistic images and videos of active combat, captured soldiers, and explosions based on user prompts.

Indirectly caused: These fakes were widely shared on social media platforms, reaching millions of users and amplifying polarizing narratives.

Inferred additional harm: Continued generation of highly deceptive military propaganda by state-linked actors to manipulate public sentiment.

Democracy

Reported: The report explicitly describes the use of AI fakes as a 'potent informational weapon' to manipulate public opinion and political processes.

Indirectly caused: The spread of state-sponsored propaganda and fake news on social media platforms undermines public trust in democratic institutions and official communications.

Inferred additional harm: Long-term erosion of democratic discourse and public trust in democratic governments' ability to manage conflicts and provide reliable information.

Epistemic

Reported: The report extensively describes the erosion of shared truth and the creation of an 'alternate reality' due to AI-generated misinformation.

Directly caused: AI systems generated over 110 unique fake images and videos depicting non-existent military events, such as the capture of US soldiers.

Indirectly caused: These fakes were viewed millions of times, leading prominent figures and the general public to believe and share false narratives about the war.

Inferred additional harm: Widespread public confusion and inability to distinguish real war documentation from synthetic 'AI slop', permanently degrading the information ecosystem.

People affected

• Occurrences reported: 1
• People reportedly exposed: 120000000

Potential causes

Management

• Prioritizing Engagement Over Safety: Platforms prioritized emotional and polarising content to drive user views.
• Inadequate Policy Enforcement: Platform management failed to enforce rules against generating warzone images.
• Premature Tech Release: AI developers released advanced tools like Sora without robust abuse prevention.

Technology

• Flawed AI Guardrails: Grok failed to block prompts about active warzones despite internal rules.
• Inconsistent Watermarking: AI watermarks were easily removed, obscured, or ignored by users.
• Generative AI Imperfections: AI tools generated physical anomalies like extra limbs and malformed hands.

Data Inputs

• Simple Text Prompts: Basic text instructions automatically generated highly detailed war imagery.
• AI Enhancement of Real Footage: Real videos were edited with AI to exaggerate explosion sizes.

Human Factors

• User Confirmation Bias: Users reposted fake images to support political views without verification.
• Rapid Mobile Consumption: Users quickly viewed content on small screens, making fakes hard to spot.
• Deliberate Disinformation Campaigns: Actors used AI-generated fakes as political propaganda to influence opinion.

Process and Methods

• Weak Platform Moderation: Social media platforms failed to proactively filter or label AI fakes.
• Bypassing Prompt Restrictions: Users easily bypassed Grok's warzone restriction from different accounts.
• Delayed Fact-Checking: Community notes and manual sweeps occurred only after fakes went viral.

Information quality

• Classification confidence: High
• Reason for confidence: The reports from reputable sources (AFP, NYT, iNews) provide consistent, detailed accounts of the proliferation of AI-generated imagery across multiple platforms, identifying specific AI tools (Grok, Gemini, Sora) and their watermarks. The evidence of guardrail bypass and the scale of dissemination is well-documented.
• Ambiguities identified: The exact proportion of fakes generated by each specific tool (Grok vs. Gemini vs. Sora) is not fully quantified, and the exact identity of all actors behind the pro-Iranian campaign remains unverified.
• Alternative interpretations: Some might interpret the incident primarily as a social media moderation failure rather than an AI safety failure, though the ease of generating realistic fakes via AI is central to the scale of the issue.

During the 2026 war in Iran, malicious actors leveraged generative AI tools to conduct a large-scale information warfare campaign. By producing realistic fakes of captured US soldiers and destroyed assets, the campaign bypassed commercial guardrails to degrade public trust and consensus reality, highlighting the evolved national security threat of AI-driven wartime propaganda.

• Overall national security impact: Substantial
• Scope: Multiple nations
• Primary target: United States
• Other affected: Iran, Israel
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Near-term. Represents an active, developing information operations campaign during an ongoing military conflict.
• Autonomy: Human-controlled. AI tools were actively prompted and directed by human operators to generate specific propaganda images and videos.
• Novelty: Evolved capability. While wartime propaganda is an established threat, the use of highly realistic, guardrail-bypassing generative AI represents a significant evolution in sophistication.

Impact by dimension

• Physical security: Negligible. No physical infrastructure damage or kinetic threats occurred; the incident was limited to digital media generation.
• Information security: Substantial. Notable information warfare impact, with highly realistic AI-generated propaganda depicting fake US military losses and missile strikes to manipulate public opinion during active conflict.
• Sovereignty: Minor. While targeting public perception during a sovereign conflict, the incident did not directly compromise core government decision-making processes or state authority.
• Economic security: Minor. Demonstrates vulnerabilities in commercial AI safety guardrails (Grok, Gemini), though it did not cause direct strategic economic damage.
• Societal stability: Substantial. Substantial erosion of consensus reality and public trust, with synthetic war media generating millions of views and spreading epistemic confusion.