Joint Base Elmendorf-Richardson Soldier Faces Allegations of Using AI to Generate Child Pornography

Seth Herrera, a U.S. Army soldier at Joint Base Elmendorf-Richardson (JBER), is accused of using artificial intelligence to generate pornography depicting minors with whom he was in contact.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.2 Exposure to toxic content

The incident involves the creation and potential proliferation of child sexual abuse material (CSAM), which is explicitly classified as toxic content under this sub-domain.

Additional risk subdomains

• 4.3 Fraud, scams, and targeted manipulation: The perpetrator targeted specific minors he knew and sexualized their likenesses using synthetic content generation.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The harm was caused by the intentional decision and action of a human user who misused deployed generative AI tools to create illegal child sexual abuse material.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. The report describes the use of generative AI and chatbots to create synthetic content and morphed images, which falls under the category of AI-generated content such as deepfakes.

AI system and alleged parties

• AI system: AI chatbots
• AI purpose: Deepfake Image Generation; Chatbot
• Behaviour type: Tool
• Alleged developer: Seth Herrera
• Alleged deployer: Unknown AI developers
• Alleged harmed parties: Children

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Minor
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Substantial, indirect Substantial

Malicious content

Reported: The report explicitly describes the generation of child sexual abuse material (CSAM) using AI.

Directly caused: The defendant used generative AI tools to morph images of real children into pornographic images depicting minors.

Indirectly caused: The defendant joined encrypted messaging groups known for trafficking CSAM, potentially distributing the generated toxic content.

Inferred additional harm: It is likely that additional toxic CSAM was generated and shared across encrypted networks, exposing more individuals to this illicit material.

Civil rights

Reported: The report explicitly describes violations of human and civil rights through the exploitation of children.

Directly caused: The fundamental rights of the minors were violated through the unauthorized creation of sexually explicit imagery depicting them.

Inferred additional harm: The rights of multiple unnamed children whose images were harvested from social media were likely violated through the creation and potential distribution of CSAM.

Privacy

Reported: The report explicitly describes privacy violations involving surreptitious recordings and harvesting social media images.

Directly caused: The defendant violated the privacy of minors by taking surreptitious recordings of them undressing in his home and harvesting their images from social media to morph them.

Inferred additional harm: The privacy of numerous other children whose images were stored in the tens of thousands of files on his phones was likely violated.

Child sexual exploitation and abuse

Reported: The report explicitly describes an AI-facilitated CSEA incident involving the generation of CSAM.

Directly caused: The defendant used AI chatbots and generative tools to morph images of real children into pornographic images depicting minors.

Indirectly caused: The AI tools facilitated the creation of realistic computer-generated CSAM, which was stored and potentially trafficked in encrypted groups.

Inferred additional harm: It is highly likely that multiple children were subjected to digital sexual exploitation and abuse through the generation of these synthetic images.

People affected

• Occurrences reported: 1
• People reportedly harmed: 2
• People reportedly exposed: 2

Potential causes

Technology

• Generative AI Image Morphing: Generative AI tools allowed morphing real children's images into CSAM.
• Encrypted Messaging Apps: Used encrypted messaging to share and traffic the illegal material.
• Password-Protected Vault Apps: Used a calculator-disguised app to conceal illegal files on devices.

Data Inputs

• Social Media Image Harvesting: Real children's images and videos were harvested from social media pages.

Human Factors

• Malicious User Intent: Defendant intentionally sought to create and view CSAM using AI.
• Technical Sophistication: Defendant possessed skills to navigate the web and hide digital footprints.

Process and Methods

• Lack of AI Safeguards: AI tools lacked sufficient safeguards to prevent generating CSAM.

Regulatory Environment

• Enforcement Challenges: Rapidly evolving AI tools make tracking and enforcement difficult.

Information quality

• Classification confidence: High
• Reason for confidence: The report is a detailed news article about a federal indictment with specific charges, quotes from officials, and factual details. The role of AI in morphing real children's images is explicitly stated.
• Ambiguities identified: The specific AI tools and chatbots used are not named.
• Alternative interpretations: None.

An active-duty U.S. Army soldier was indicted for using generative AI tools to create child sexual abuse material from social media photos. While representing a severe criminal act and violation of individual human rights, the incident has minor national security implications, as it was a localized criminal matter handled by law enforcement without broader threat to national infrastructure, intelligence, or sovereignty.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Individual (U.S. Army soldier)

Threat characteristics

• Imminence: Long-term. The suspect has been arrested and detained, resolving any immediate crisis. The threat represents an ongoing strategic concern regarding the misuse of consumer AI tools.
• Autonomy: Human-controlled. The AI tools were directly operated by the human user to generate and morph specific images, acting purely as a tool without autonomous decision-making.
• Novelty: Evolved capability. The case represents an evolution of existing CSAM threats, utilizing widely available generative AI tools to morph real images into synthetic abusive content.

Impact by dimension

• Physical security: Negligible. The incident involves criminal activity by an individual soldier utilizing generative AI for illicit image creation, presenting no threat to physical systems or critical infrastructure.
• Information security: Negligible. No classified networks, intelligence collection methods, or state-sponsored information warfare operations were involved in this incident.
• Sovereignty: Minor. While the perpetrator is an active-duty military service member, the incident is a personal criminal matter being handled through standard legal and military justice procedures, with minor impact on government operations.
• Economic security: Negligible. There is no indication of strategic technology theft, financial system disruption, or loss of national technological competitive advantage.
• Societal stability: Minor. The incident represents a severe violation of individual civil and human rights of the targeted minors, but does not threaten population-scale societal stability or cause widespread civil unrest.