Iranian Hacker Group Cotton Sandstorm Reportedly Integrating AI into Cyber Influence Operations

The Iranian state-sponsored group Cotton Sandstorm, linked to the IRGC, has integrated generative AI into cyber influence operations. In December 2023, it launched Operation “For Humanity," using AI-crafted messaging to hijack a U.S.-based IPTV streaming service with propaganda about the Israel-Hamas conflict. The group also engages in election-related reconnaissance, which suggests they used AI-enhanced influence efforts ahead of the 2024 U.S. election.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale

The incident involves state-sponsored actors using AI systems to conduct coordinated, large-scale disinformation campaigns aimed at manipulating public opinion and political processes.

Additional risk subdomains

• 3.2 Pollution of information ecosystem and loss of consensus reality: The creation of hundreds of fake news websites and automated social media comments polluted the online information ecosystem.
• 1.2 Exposure to toxic content: The campaigns involved generating and spreading highly deceptive deepfakes and false allegations of sexual assault to denigrate political candidates.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The risk was caused by the intentional decisions and actions of human threat actors who deployed already-existing AI models to conduct disinformation campaigns.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the use of generative AI to create deepfakes of political figures (e.g., 'AI-enhanced deepfake videos about Vice President Kamala Harris') and generate text/images, which fall under the transparency obligations of Limited Risk systems.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Social Media Content Generation; Deepfake Video Generation
• Behaviour type: Assistant
• Alleged developer: Unknown generative AI developers, Islamic Revolutionary Guard Corps (IRGC), Government of Iran, Cotton Sandstorm
• Alleged deployer: Islamic Revolutionary Guard Corps (IRGC), Government of Iran, Cotton Sandstorm
• Alleged harmed parties: U.S. elections, political candidates, Media organizations, General public of the United States, Electoral integrity, Democracy, American voters

Harm severity

Highest direct severity in any category: Severe. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Severe, indirect Negligible
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Minor, indirect Negligible
• Privacy: direct Substantial, indirect Negligible
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Substantial, indirect Substantial
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes, the reports describe the creation and spread of deepfake videos, fake news articles, and antisemitic messages.

Directly caused: Russian group Storm-1516 created deepfake videos of Kamala Harris and Tim Walz (including fake sexual assault allegations). Chinese group Spamouflage spread antisemitic messages and political cartoons.

Indirectly caused: N/A

Inferred additional harm: It is likely that many more toxic or malicious posts and comments were generated by AI tools across various social media platforms that went undetected.

Differential treatment

Reported: Yes, the reports mention Chinese operations targeting specific Republican candidates who support Israel with antisemitic language.

Directly caused: Spamouflage targeted Jewish or pro-Israel politicians (like Barry Moore) using antisemitic language.

Indirectly caused: N/A

Inferred additional harm: N/A

Democracy

Reported: Yes, the reports explicitly state that Russia, Iran, and China sought to undermine Americans' faith in the democratic process and the integrity of the election.

Directly caused: Adversaries used AI-generated content, deepfakes, and fake news websites to spread claims of voter fraud and denigrate candidates, aiming to undermine public trust in the 2024 U.S. election.

Indirectly caused: N/A

Inferred additional harm: Widespread exposure to these campaigns may have incrementally eroded public trust in democratic institutions and election outcomes among some segments of the population.

Privacy

Reported: Yes, the reports mention Emennet Pasargad hacking databases to steal personal information and harvesting data from IP cameras.

Directly caused: Emennet Pasargad stole personal information of 200,000 customers of French magazine Charlie Hebdo and harvested data from IP cameras.

Indirectly caused: N/A

Inferred additional harm: Threat actors likely compromised the privacy of numerous other individuals through spear-phishing and OSINT harvesting on ancestry and social media sites.

Psychological

Reported: Yes, the reports describe Emennet Pasargad attempting to contact family members of Israeli hostages to cause psychological trauma.

Directly caused: Iranian threat actors sent messages to families of Israeli hostages to cause additional psychological effects and inflict further trauma.

Indirectly caused: N/A

Inferred additional harm: The targeted harassment and deepfakes of political figures likely caused personal distress, though not explicitly quantified.

Epistemic

Reported: Yes, the reports describe the creation of fake news websites, fabricated articles, and cloned voices to spread false claims.

Directly caused: CopyCop created over 160 fake websites using ChatGPT to rewrite and plagiarize articles. Operation Overload used AI to clone the FBI director's voice to fabricate claims of voter fraud.

Indirectly caused: These operations tricked fact-checking organizations into spreading their narratives over 250 times, injecting false claims into the broader information ecosystem.

Inferred additional harm: The proliferation of AI-generated fake news sites and social media comments likely contributed to a general degradation of the online information ecosystem, making it harder for users to distinguish real news from synthetic content.

People affected

• Occurrences reported: 1
• People reportedly harmed: 50
• People reportedly exposed: 5000000

Potential causes

Management

• State-Sponsored Funding of IOs: Adversary governments financed front companies to run AI campaigns.
• Use of Front Companies: Emennet Pasargad used ASA as a cover to access commercial AI tools.

Technology

• AI Voice Cloning Tools: Used to clone the FBI director's voice to fabricate voter fraud claims.
• Generative AI Text Models: ChatGPT used to rewrite plagiarized articles and generate fake comments.
• AI Deepfake Video Generators: Created manipulated videos of political candidates to damage reputations.

Data Inputs

• Plagiarized Media Content: Generative AI used stolen news articles as input to rewrite propaganda.
• Leaked Personal Datasets: Ancestry and leaked data used to target specific military personnel.
• Open-Source Information Harvesting: Adversaries scraped LinkedIn and ancestry sites to identify target personnel.

Human Factors

• Targeted Phishing Susceptibility: Campaign staff fell victim to spear-phishing, leaking campaign data.
• Audience Trust in Audio Visuals: Voters easily believe realistic AI-cloned voices and deepfake videos.

Process and Methods

• Automated Content Generation: AI tools enabled rapid scaling of low-quality propaganda across platforms.
• Inauthentic Account Automation: Automation scripts deployed fake accounts to amplify AI-generated content.
• Typosquatting and Redirects: Redirect techniques bypassed platform moderators to show fake AI news.

Regulatory Environment

• Lax Identity Verification: Hosting providers allowed anonymous purchase of server space by adversaries.
• Inadequate Platform Moderation: Social media platforms struggled to proactively block AI-generated fakes.

Information quality

• Classification confidence: High
• Reason for confidence: Multiple independent sources, including Microsoft, the FBI, DOJ, CISA, and Recorded Future, provide highly detailed, consistent, and corroborated accounts of the threat actors' activities and their use of AI. The role of AI is explicitly detailed across multiple reports, leaving little ambiguity about the nature of the operations.
• Ambiguities identified: The exact impact of the AI-generated content on individual voters' decisions is difficult to measure quantitatively.
• Alternative interpretations: The operations could be viewed primarily as traditional cyber espionage and phishing campaigns, with AI playing only a minor, secondary role in content generation.

State-sponsored actors from Russia, Iran, and China leveraged generative AI to scale disinformation and influence operations targeting the 2024 U.S. election and geopolitical targets. While the operations were successfully mitigated and had limited persuasive impact, they highlight an evolved threat to information security, democratic sovereignty, and societal stability.

• Overall national security impact: Substantial
• Response level: Substantial
• Scope: Multiple nations
• Primary target: United States
• Other affected: Israel, France, Sweden
• Alleged perpetrator: State-sponsored threat actors from Russia, Iran, and China

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern and persistent capability development by foreign adversaries targeting democratic institutions.
• Autonomy: Human-controlled. AI systems were used as tools to assist human actors in content generation, requiring human direction and prompt engineering.
• Novelty: Evolved capability. Represents a significant technological advancement of traditional foreign influence operations through the integration of generative AI.

Impact by dimension

• Physical security: Minor. Minor cyber compromises occurred against a French display provider and US IPTV services, but no physical damage, kinetic threats, or critical infrastructure disruptions were reported.
• Information security: Substantial. State-sponsored groups integrated generative AI to scale disinformation, clone the FBI director's voice, and generate deepfakes of political candidates, warranting serious intelligence response.
• Sovereignty: Substantial. Foreign adversaries used AI-driven campaigns to target the 2024 US presidential election and undermine democratic processes, though coordinated defense successfully mitigated the impact.
• Economic security: Minor. No critical financial systems or strategic technologies were compromised, though substantial resources were expended by government and private entities to counter the operations.
• Societal stability: Substantial. Campaigns specifically sought to deepen domestic political polarization, spread antisemitic content, and harass families of hostages, threatening social cohesion.