ISIS Utilizes AI for Propaganda Videos in News Harvest Program

ISIS supporters have created an AI-generated media program called News Harvest to disseminate propaganda videos. The program produces near-weekly broadcasts featuring AI-generated news anchors discussing ISIS operations globally, using cheap and easy-to-use AI tools. This development showcases the use of AI as a powerful propaganda tool for extremist groups.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale

Extremist organizations are using AI to conduct coordinated, large-scale propaganda and influence campaigns aimed at manipulating public opinion and recruiting followers.

Additional risk subdomains

• 1.2 Exposure to toxic content: The AI tools are being used to generate and spread highly toxic content, including extremist propaganda, hate speech, and bomb-making instructions.
• 4.3 Fraud, scams, and targeted manipulation: Far-right extremists are using voice cloning and deepfake technologies to impersonate public figures for targeted manipulation and recruitment.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The risk is directly caused by human extremists intentionally utilizing deployed AI systems to generate propaganda and bypass safety guardrails.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. The reports describe the use of 'AI-generated content such as deepfakes' and 'chatbots' which require transparency obligations under the Act.

AI system and alleged parties

• AI system: ChatGPT, ElevenLabs, unspecified AI program (ElevenLabs, OpenAI)
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: OpenAI, ElevenLabs
• Alleged deployer: ISIS supporters, ISIS
• Alleged harmed parties: People susceptible to radicalism, General public

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Minor
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Minor, indirect Negligible
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Minor

Malicious content

Reported: The report explicitly describes the creation and spread of extremist propaganda, deepfakes, and bomb-making instructions using AI.

Directly caused: AI tools were used to generate 'News Harvest' videos, cloned voices of public figures, and sectarian/antisemitic memes.

Indirectly caused: The dissemination of these materials across Telegram, Rocket.Chat, and other platforms reached thousands of users.

Inferred additional harm: It is highly likely that significantly more extremist content was generated and circulated across private channels than what was captured by SITE Intelligence Group.

Differential treatment

Reported: The report explicitly describes the use of AI to spread antisemitic propaganda and target Jewish people.

Directly caused: AI-generated deepfakes and graphics were used to promote antisemitic narratives and target specific religious/ethnic groups.

Inferred additional harm: Widespread dissemination of AI-generated hate speech likely contributed to increased hostility and discrimination against Jewish and other minority communities online.

Democracy

Reported: The report explicitly describes the use of AI to spread coordinated disinformation and propaganda to manipulate public opinion.

Directly caused: Extremists used AI to create fake news broadcasts ('News Harvest') and deepfakes of political figures to influence public perception.

Inferred additional harm: The proliferation of highly realistic AI-generated extremist propaganda threatens to erode public trust in legitimate news media and democratic institutions.

Privacy

Reported: The report explicitly describes voice cloning of public figures without their consent.

Directly caused: ElevenLabs was used to clone the voices of individuals (like Jordan Peterson) without their consent to read extremist texts.

Inferred additional harm: Other private individuals may have had their voices or likenesses cloned for targeted harassment or manipulation.

Epistemic

Reported: The report explicitly describes the creation of fake news broadcasts ('News Harvest') and deepfakes of public figures.

Directly caused: AI was used to fabricate news anchors and clone voices to present extremist propaganda as legitimate news.

Inferred additional harm: The widespread availability of these tools makes it increasingly difficult for the public to distinguish between authentic and synthetic news, eroding shared reality.

Child sexual exploitation and abuse

Reported: The report mentions a neo-Nazi network (GDL) notorious for 'hate-grooming children online' using AI-generated content.

Indirectly caused: AI-generated videos and memes were used by GDL to attract and radicalize children online.

Inferred additional harm: It is possible that children targeted by these networks experienced online grooming facilitated by AI-generated recruitment materials.

People affected

• Occurrences reported: 1
• People reportedly exposed: 5000

Potential causes

Management

• Reckless Tech Company Governance: AI companies shirk responsibility and fail to proactively block bad actors.
• Lack of Proactive Risk Assessment: Companies release powerful tools to the public without adequate safety fences.

Technology

• Highly Accessible Generative AI: Cheap, easy tools like ElevenLabs and Wav2Lip enable rapid media creation.
• Bypassable Chatbot Guardrails: Users easily sidestep safety protocols with simple social engineering prompts.
• High-Fidelity Translation Tools: AI allows extremist materials to be translated with chilling accuracy.

Data Inputs

• Extremist Text as Prompts: Official ISIS dispatches and Mein Kampf are fed directly into AI generators.
• Lack of Input Sanitization: AI systems process extremist literature without blocking harmful source texts.

Human Factors

• Proactive Extremist Adoption: Extremists quickly adopt new technologies to gain strategic advantages.
• Social Engineering of Chatbots: Users trick AI by claiming to be police officers or certified experts.

Process and Methods

• Inadequate Content Moderation: Platforms like Rocket.Chat, Telegram, and X fail to remove extremist media.
• Innocuous Packaging of Propaganda: AI-generated news style hides extremist content, making moderation harder.

Regulatory Environment

• Lagging Government Regulations: Lawmakers and regulators cannot keep pace with rapidly evolving AI tech.
• Under-Informed Regulators: Regulators lack the expertise to address AI-driven extremist threats.

Information quality

• Classification confidence: High
• Reason for confidence: The reports are highly detailed and consistent, drawing from credible monitoring organizations like SITE Intelligence Group and MEMRI. They provide specific examples of AI tools, platforms, and extremist groups involved.
• Ambiguities identified: The exact AI models used for some of the image and video generation are not specified, though ElevenLabs and ChatGPT are explicitly named.

Extremist and terrorist organizations are successfully exploiting consumer generative AI tools to lower the cost and accelerate the scale of propaganda, deepfakes, and recruitment. While not representing an immediate kinetic crisis, this evolved capability significantly enhances information warfare and societal polarization threats globally.

• Overall national security impact: Substantial
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Global
• Alleged perpetrator: Extremist groups including ISIS, al-Qaeda, and neo-Nazi networks

Threat characteristics

• Imminence: Long-term. Represents an ongoing, strategic shift in extremist capabilities and recruitment methodologies rather than an immediate, single-point crisis.
• Autonomy: Human-controlled. AI systems are used directly as tools by human operators who prompt and direct the generation of content.
• Novelty: Evolved capability. Significant advancement of traditional extremist propaganda and recruitment methods through the integration of consumer generative AI.

Impact by dimension

• Physical security: Minor. Extremists bypassed chatbot safety guardrails to obtain bomb-making and explosives instructions, posing a minor physical security concern.
• Information security: Substantial. Terrorist and extremist groups are actively utilizing generative AI, voice cloning, and synthetic news anchors to scale propaganda and influence operations.
• Sovereignty: Minor. Minor risk to state authority through accelerated radicalization and recruitment efforts aimed at destabilizing democratic societies.
• Economic security: Negligible. No significant threats to financial systems, critical supply chains, or strategic technological infrastructure reported.
• Societal stability: Substantial. Extremist networks are leveraging AI to scale antisemitic hate campaigns and target vulnerable populations, including the online hate-grooming of children.