Far-Right Teen in Singapore Reportedly Queried AI Chatbot About Ammunition While Planning Mosque Attacks

Singapore authorities reported that a 17-year-old far-right extremist searched an unidentified AI chatbot for ammunition-making instructions while preparing attacks on Muslims at five mosques. He reportedly aimed to kill at least 100 people but failed to obtain a firearm and was detained before carrying out an attack. The public sources do not establish what the chatbot answered or whether its output aided his preparations.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.2 Exposure to toxic content

The ISIS chatbot exposed the minor to violent extremist ideology and facilitated her taking a pledge of allegiance to a terrorist organization, which constitutes exposure to toxic, extremist, and illegal content.

Additional risk subdomains

• 4.1 Disinformation, surveillance, and influence at scale: The chatbot was utilized by a terrorist organization as an automated tool to conduct influence and recruitment campaigns targeting vulnerable individuals.

Causal factors

• Entity: AI
• Intent: Intentional
• Timing: Post-deployment

The radicalization process was facilitated by an active ISIS chatbot deployed online, which intentionally engaged the user and accepted her pledge of allegiance.

EU AI Act risk tier

• Risk tier: 1 Unacceptable

Unacceptable Risk: The chatbot was used for subliminal manipulation and behavioral influence leading to the severe radicalization of a vulnerable minor.

AI system and alleged parties

• AI system: unspecified AI chatbot
• AI purpose: Chatbot; Content Recommendation
• Behaviour type: Assistant
• Alleged developer: Large language model developers, Chatbot developers
• Alleged deployer: Terrorism suspects, Men and boys, Malicious actors, Far-right extremists, Extremists, Chatbot users, Boys, Adolescents, Adolescent boys
• Alleged harmed parties: Religious communities, Muslims, General public of Singapore, General public

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Minor
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the creation and spread of toxic and malicious extremist content.

Directly caused: The chatbot accepted a virtual pledge of allegiance to a terrorist organization, validating extremist beliefs.

Indirectly caused: Following her interaction, the girl actively reposted extremist materials, including ISIS chants, on her social media accounts and shared them with schoolmates.

Inferred additional harm: The chatbot likely generated and disseminated extremist propaganda to numerous other users online.

Civil rights

Reported: The report explicitly describes a situation resulting in the restriction of civil liberties.

Directly caused: The girl's radicalization via the chatbot led to her being issued a Restriction Order under the Internal Security Act, severely limiting her freedom of movement, internet access, and association.

Psychological

Reported: The report explicitly describes psychological harm in the form of severe radicalization of a minor.

Directly caused: The 15-year-old girl experienced cognitive and emotional radicalization, leading her to pledge allegiance to a violent terrorist group and desire to die as a martyr.

Inferred additional harm: It is highly likely that other individuals interacting with the same chatbot suffered similar psychological manipulation and radicalization.

Epistemic

Reported: The report explicitly describes epistemic harm where the user was convinced of false, extremist narratives.

Directly caused: The chatbot reinforced the false belief that ISIS was a legitimate religious army defending Muslims, corrupting the user's understanding of reality.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 1

Potential causes

Technology

• Interactive ISIS Chatbot: Facilitated a virtual pledge of allegiance, solidifying extremist identity.
• Recommendation Algorithms: Served extremist propaganda and violent videos, accelerating radicalisation.
• Search Engine Surfacing: Allowed easy retrieval of extremist manifestos and potential target mosques.

Data Inputs

• Extremist Chatbot Training: Chatbot was fed extremist data, enabling it to accept pledges to ISIS.
• Unmoderated Video Feeds: Algorithmic feeds ingested and distributed edited Christchurch attack videos.
• Online Terrorist Manifestos: Digitised manifestos served as inputs for youth self-radicalisation.

Human Factors

• Youth Susceptibility to AI: Vulnerable youth formed emotional or ideological bonds with an AI chatbot.
• Parental Inaction: Parents noticed excessive online activities but did not report to authorities.
• Peer Reporting Failure: Schoolmates ignored shared extremist materials and did not alert teachers.

Process and Methods

• Flawed Platform Moderation: Online platforms failed to detect and block the interactive ISIS chatbot.
• Algorithmic Feed Delivery: Automated distribution processes prioritized engagement with radical content.

Information quality

• Classification confidence: High
• Reason for confidence: The reports are official government releases from Singapore's Internal Security Department and credible news coverage, providing clear facts about the youth's interaction with the chatbot.
• Ambiguities identified: The specific platform hosting the chatbot and its technical architecture are not detailed.
• Alternative interpretations: The chatbot could be interpreted as a passive tool rather than an active agent of radicalization, though its interactive nature supports the classification.

A 15-year-old Singaporean girl was radicalized and took a virtual pledge of allegiance to an ISIS-run chatbot, leading to her restriction under the Internal Security Act. This incident highlights a highly novel and concerning precedent where extremist groups leverage autonomous AI to scale, automate, and facilitate recruitment and radicalization of vulnerable individuals.

• Overall national security impact: Substantial
• Scope: Single nation
• Primary target: Singapore
• Alleged perpetrator: ISIS

Threat characteristics

• Imminence: Long-term. Represents an ongoing, strategic threat of automated online radicalization rather than an immediate kinetic crisis.
• Autonomy: Full autonomy. The chatbot interacted with the user and accepted her pledge of allegiance autonomously without real-time human intervention.
• Novelty: First-of-its-kind. First recorded instance of an individual taking a virtual pledge of allegiance (bai'ah) directly to an AI chatbot representing a terrorist group.

Impact by dimension

• Physical security: Minor. The minor expressed willingness to travel to Syria to engage in combat, demonstrating a potential path to physical violence, though intercepted before execution.
• Information security: Substantial. Use of an AI chatbot by a terrorist organization to automate recruitment, propaganda, and obtain pledges of allegiance represents a substantial intelligence and information security challenge.
• Sovereignty: Minor. Required intervention by Singapore's Internal Security Department under the Internal Security Act, though core government operations were not disrupted.
• Economic security: Negligible. No significant implications for financial systems, strategic technologies, or economic security were reported.
• Societal stability: Minor. The incident involved the successful radicalization of a minor and subsequent restriction of her civil liberties under national security laws.