A young man was arrested in Cornella, Spain, for using AI-powered editing applications to create and distribute jihadist propaganda to radicalize others online.
Spanish authorities alleged that a suspect arrested in Cornellà, near Barcelona, used AI-supported editing applications to produce jihadist multimedia propaganda. He allegedly distributed material through social media and encrypted messaging to radicalize others and promote terrorist violence. A court ordered pretrial detention. The reports do not identify the applications or establish that his activity led to an attack.
Risk classification
- Primary risk domain: 4 Malicious actors
- Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale
The detainee used AI tools to generate and spread jihadist propaganda at scale to influence and radicalize his large online following.
Additional risk subdomains
- 1.2 Exposure to toxic content: The AI-modified media was used to expose online users to extremist, violent, and terrorist content.
Causal factors
- Entity: Human
- Intent: Intentional
- Timing: Post-deployment
The risk was caused by a human actor intentionally using deployed AI-powered editing tools to create and distribute extremist propaganda.
EU AI Act risk tier
- Risk tier: 3 Limited Risk
Limited Risk: The report describes the use of AI to generate or modify multimedia content, which falls under transparency obligations for synthetic media.
AI system and alleged parties
- AI system: AI-supported editing applications
- AI purpose: Image Retouching; Social Media Content Generation
- Behaviour type: Tool
- Alleged developer: Synthetic media technology developers, AI-assisted multimedia editing tool developers
- Alleged deployer: Terrorism suspects, Synthetic media creators, Extremists
- Alleged harmed parties: Social media users, People susceptible to radicalism
Harm severity
Severity ratings are based on this scale.
Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.
- Physical: direct Negligible, indirect Negligible
- Infrastructure: direct Negligible, indirect Negligible
- Property: direct Negligible, indirect Negligible
- Financial: direct Negligible, indirect Negligible
- Environmental: direct Negligible, indirect Negligible
- Malicious content: direct Substantial, indirect Negligible
- Differential treatment: direct Negligible, indirect Negligible
- Civil rights: direct Negligible, indirect Negligible
- Democracy: direct Negligible, indirect Negligible
- Privacy: direct Negligible, indirect Negligible
- Psychological: direct Negligible, indirect Negligible
- Epistemic: direct Negligible, indirect Negligible
- Child sexual exploitation and abuse: direct Negligible, indirect Negligible
Malicious content
Reported: The report explicitly describes toxic and malicious content created and spread directly by the suspect using AI.
Directly caused: The suspect used AI-powered editing applications to format and modify images to create jihadist propaganda.
Inferred additional harm: It is likely that his numerous online followers were exposed to this toxic extremist content before his arrest.
People affected
- Occurrences reported: 1
- People reportedly exposed: 1
Potential causes
Technology
- AI-Powered Media Editing: AI tools enabled rapid creation and modification of terrorist propaganda.
- Encrypted Messaging Channels: Encrypted platforms prevented easy detection of shared AI propaganda.
Data Inputs
- Source Image Manipulation: Using existing images as inputs for AI modification to create propaganda.
Human Factors
- Susceptibility to AI Content: Followers were easily influenced by sophisticated AI-generated media.
- Tech-Savvy Extremist Actors: A young operator utilized AI tools to maximize his online influence.
Process and Methods
- Digital Propaganda Workflows: Streamlined processes using AI to format and distribute extremist media.
Regulatory Environment
- Unregulated AI Editing Tools: Lack of regulation preventing AI tools from generating extremist content.
Information quality
- Classification confidence: Medium
- Reason for confidence: The reports clearly state that the suspect used AI-powered editing applications to modify images and create propaganda. However, the specific AI models, software names, and exact scale of dissemination are not detailed, requiring some inference regarding the reach of the content.
- Ambiguities identified: The exact nature of the AI-powered editing applications and how they were used to modify the images is not specified.
- Alternative interpretations: The AI tools could be viewed as standard consumer software with minor AI features, making the incident primarily a human-driven terrorist propaganda case rather than an AI-centric safety failure.
- Missing information: The specific names of the AI applications used, the exact number of followers exposed, and details on how the AI modified the images.
Spanish authorities arrested a Moroccan national in Spain for using AI-powered editing applications to create and distribute jihadist propaganda. The incident highlights the growing use of consumer AI tools by malicious actors to enhance extremist content creation, though the overall national security impact remains minor due to swift law enforcement intervention.
- Overall national security impact: Minor
- Scope: Single nation
- Primary target: Spain
- Alleged perpetrator: Individual (Moroccan national)
Threat characteristics
- Imminence: Long-term. The suspect was arrested and placed in provisional prison, neutralizing the immediate threat, leaving only long-term strategic concerns.
- Autonomy: Human-controlled. The AI systems (editing applications) were used strictly as tools directed and controlled by the human suspect.
- Novelty: Evolved capability. Using digital tools for propaganda is established, but leveraging consumer AI editing tools represents an evolved capability in content creation.
Impact by dimension
- Physical security: Negligible. No physical infrastructure, targeting systems, or kinetic attacks were involved or compromised.
- Information security: Minor. The suspect used AI tools to modify images for jihadist propaganda, representing a low-level, localized information operation rather than a state-sponsored campaign.
- Sovereignty: Negligible. No core government decisions, electoral systems, or constitutional processes were disrupted or compromised.
- Economic security: Negligible. No financial systems, strategic technology, or critical supply chains were targeted or affected.
- Societal stability: Minor. The creation and distribution of extremist propaganda aimed to radicalize individuals, posing a minor threat to societal cohesion, though mitigated by law enforcement intervention.