Purportedly AI-Generated Images Reportedly Spread Misinformation During Hurricane Helene Response

During Hurricane Helene (September 24-29, 2024), purportedly AI-generated images circulated on social media, reportedly misleading the public and hindering disaster response efforts. Reportedly fake images including animals stranded on rooftops and political figures in floodwaters added confusion and disrupted emergency management. The spread of these images allegedly exacerbated existing challenges, including power outages and communication failures, all of which led to complications providing accurate information to those in need.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.2 Pollution of information ecosystem and loss of consensus reality

The widespread flooding of social media with AI-generated disaster images led to 'verification fatigue' and public apathy, severely degrading the shared information ecosystem during a real-world crisis.

Additional risk subdomains

• 4.1 Disinformation, surveillance, and influence at scale: The synthetic images were actively used by political influencers and campaigns as propaganda to shape public opinion and polarize narratives regarding the government's disaster response.
• 4.3 Fraud, scams, and targeted manipulation: The reports highlight that these AI-generated images are used by malicious actors as deceptive hooks for phishing schemes and fraudulent fundraising campaigns.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The risk and subsequent harms were caused by human actors intentionally generating and sharing deceptive AI images to spread propaganda, gain social media engagement, or facilitate scams.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. The reports describe the generation and dissemination of 'deepfake' images and AI-generated content, which are subject to transparency obligations under the EU AI Act to ensure users are aware they are interacting with synthetic media.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Deepfake Image Generation; Social Media Content Generation
• Behaviour type: Tool
• Alleged developer: Unknown deepfake technology developers, Unknown AI image generator technology developers
• Alleged deployer: Unknown malicious actors, Unknown deepfake creators, Unknown actors, Partisan social media influencers
• Alleged harmed parties: Hurricane Helene victims, General public, Emergency responders, Communities impacted by Hurricane Helene

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Minor
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Minor, indirect Minor
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Minor, indirect Minor
• Epistemic: direct Substantial, indirect Substantial
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the creation and spread of deceptive, emotionally manipulative deepfake images of suffering children and animals.

Directly caused: AI-generated deepfakes depicting a crying girl holding a puppy in floodwaters and animals stranded on roofs were widely circulated across social media platforms.

Indirectly caused: The spread of these deceptive images fueled online conspiracy theories, such as claims that the government controlled the weather to target specific voters.

Inferred additional harm: Millions of social media users were exposed to emotionally manipulative synthetic media designed to provoke strong negative reactions.

Democracy

Reported: The report explicitly describes the erosion of public trust in government institutions and the use of fake images as political propaganda.

Directly caused: Right-wing influencers and politicians used the synthetic images as political propaganda to attack the administration's disaster response, undermining institutional trust.

Indirectly caused: The spread of misinformation forced emergency response agencies like FEMA to divert critical resources to establish a 'Rumor Response' page to counter false narratives.

Inferred additional harm: The systematic use of AI-generated content to polarize public opinion during disasters threatens the stability of democratic communication and emergency management.

Psychological

Reported: The report explicitly describes psychological impacts including emotional distress, confusion, anger, and verification fatigue caused by the deepfakes.

Directly caused: Users experienced 'emotional whiplash'—initial shock and sadness followed by feelings of betrayal, confusion, and anger upon learning the images were fake.

Indirectly caused: Repeated exposure to fake crisis content led to 'disaster fatigue' and emotional desensitization, chipping away at people's empathy and willingness to engage with real crises.

Inferred additional harm: Thousands of social media users likely suffered from cognitive fatigue and emotional detachment due to the overwhelming presence of deceptive content.

Epistemic

Reported: The report extensively describes epistemic harm, including the erosion of shared truth, public trust in legitimate news, and the pollution of the information ecosystem.

Directly caused: The viral spread of fake disaster images led users to question all media, including accurate and necessary disaster updates.

Indirectly caused: Widespread 'verification fatigue' caused individuals to disengage from news entirely, weakening the collective ability to coordinate and respond to real crises.

Inferred additional harm: A systemic decline in the public's capacity to establish consensus reality during emergencies, leading to long-term skepticism toward all factual reporting.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1000
• People reportedly exposed: 1000000

Potential causes

Management

• Engagement Monetization Models: Platforms pay content farms based on engagement, incentivizing fake images.
• Prioritizing Engagement Over Truth: Platform management altered algorithms to favor viral reach over accuracy.

Technology

• Advanced Generative AI: Cheap and easy tools create highly realistic, deceptive synthetic images.
• Algorithmic Amplification: Social media algorithms prioritize highly emotional and engaging fake content.
• Physical Anomalies in AI Output: Generators produce subtle errors like extra fingers or mismatched colors.

Data Inputs

• Lack of Image Provenance: Uploaded images lack verifiable metadata to trace their synthetic origin.
• Exploitative Prompting: Users feed emotionally charged prompts to generate shocking disaster scenes.

Human Factors

• Verification Fatigue: The mental effort to verify images leads users to disengage and stop caring.
• Emotional Reactivity: Arresting visual images evoke strong emotions before cognitive processing.
• Lack of Digital Literacy: Many users cannot identify subtle physical inconsistencies in synthetic media.

Process and Methods

• Weakened Platform Moderation: Changes in verification and moderation allow unverified deepfakes to spread.
• Inadequate Verification Speed: Manual fact-checking is too slow to counter the viral spread of AI images.
• Erosion of Local News: Shrinking local news creates an information vacuum filled by fake AI content.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed, and corroborating accounts of the specific viral AI-generated images, their visual flaws, the public figures who shared them, and the resulting societal and psychological impacts. There is high consensus across the sources regarding the nature of the incident.
• Ambiguities identified: The specific generative AI models used to create the images are not identified, and the exact number of individuals affected by associated phishing scams is not quantified.
• Alternative interpretations: None. The consensus across all reports is that the images are AI-generated fakes that caused public confusion.

The incident highlights the national security risks of generative AI being used to pollute the information ecosystem during a domestic natural disaster. While not a kinetic threat, the widespread dissemination of realistic deepfakes eroded public trust in government agencies, forced federal emergency responders to divert resources, and demonstrated an evolved capability for exploiting crises to drive societal polarization.

• Overall national security impact: Substantial
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding the weaponization of generative AI tools to exploit public emergencies and polarize societal trust.
• Autonomy: Human-controlled. AI systems functioned strictly as tools, generating static synthetic images based on direct human prompts without autonomous decision-making.
• Novelty: Evolved capability. An evolved capability where hyper-realistic generative text-to-image models allow low-skill actors to rapidly manufacture highly convincing propaganda at scale during active crises.

Impact by dimension

• Physical security: Minor. Minor impact on emergency response operations as federal emergency management resources (FEMA) had to be diverted to counter online misinformation and coordinate truth-based disaster relief.
• Information security: Substantial. Substantial impact on the information ecosystem, where highly realistic deepfakes polluted public discourse, fueled partisan narratives, and eroded trust in official communications during a crisis.
• Sovereignty: Minor. Minor disruption to government functions, with federal agencies forced to establish dedicated rumor control operations to maintain institutional authority and counter political propaganda.
• Economic security: Minor. Minor economic threat, primarily limited to opportunistic cybercriminals leveraging synthetic media for fraudulent fundraising campaigns and phishing scams targeting the public.
• Societal stability: Substantial. Substantial societal impact by inducing verification fatigue and public apathy, which erodes social cohesion and reduces the public's willingness to assist in real-world humanitarian crises.