Reported AI Impersonations of Pastors Used in Online Donation and Influence Scams

Religious communities in the U.S. and abroad reported the circulation of purportedly AI-generated videos and cloned audio impersonating pastors and church leaders, including Father Mike Schmitz, to solicit donations and spread incendiary sermons. The impersonations reportedly leveraged synthetic voice and likeness tools and were distributed via platforms such as TikTok, Facebook, and messaging apps. Some content remained unlabeled as AI-generated and was treated as authentic by viewers.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The primary harm involves scammers using AI-generated deepfakes and voice clones to impersonate trusted religious figures for financial gain and targeted manipulation.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Vulnerable users are relying on religious chatbots, which can reinforce delusions and exacerbate mental health issues like psychosis.
• 3.1 False or misleading information: AI-generated sermons and videos are spreading false information that users mistake for genuine religious guidance.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was primarily caused by human malicious actors intentionally deploying AI deepfakes and voice clones to scam churchgoers.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3. Limited Risk: AI systems that pose a moderate risk and require specific transparency obligations... Examples: Chatbots, AI-generated content such as deepfakes.

AI system and alleged parties

• AI system: ChatGPT, unspecified (OpenAI)
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: Unknown voice cloning technology developers, Unknown deepfake technology developers
• Alleged deployer: Unknown scammers impersonating religious figures, Unknown scammers
• Alleged harmed parties: Religious leaders, Religious institutions, Religious communities, Mike Schmitz, General public, Epistemic integrity, Congregants, Churches

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Severe. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Minor
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Substantial
• Epistemic: direct Severe, indirect Severe
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the creation and spread of malicious deepfakes and fake videos impersonating religious figures.

Directly caused: Scammers directly created deepfake videos of Father Schmitz and other priests urging viewers to click fraudulent links.

Indirectly caused: An AI pastor TikTok account posted synthetic sermons that went viral, racking up over 11 million views without indicating they were AI-generated.

Inferred additional harm: It is likely that many more unrecorded deepfakes and synthetic religious videos are circulating on social media platforms, exposing millions of users to deceptive content.

Psychological

Reported: The report explicitly describes concerns about users experiencing severe mental health consequences, such as reinforcing religious delusions and psychosis.

Indirectly caused: The report notes that OpenAI reported hundreds of thousands of ChatGPT users showing signs of psychosis and other mental health issues, with some delusions being religious in nature and reinforced by AI.

Inferred additional harm: It is likely that hundreds of vulnerable users experiencing religious delusions have had their mental health conditions worsened by interacting with chatbots that reinforce their beliefs.

Epistemic

Reported: The report explicitly describes epistemic harm through the generation of fake religious sermons and deceptive videos that users believed were real.

Directly caused: An AI-generated video of a pastor yelling about billionaires received 11 million views, with most commenters treating it as real and expressing shocked beliefs.

Indirectly caused: The proliferation of AI-generated religious content makes it difficult for followers to distinguish between real and fake statements from religious authorities.

Inferred additional harm: The widespread use of AI to fabricate religious authority figures likely erodes trust in religious institutions and degrades the shared information ecosystem for believers.

People affected

• Occurrences reported: 1
• People reportedly harmed: 10
• People reportedly exposed: 12200000

Potential causes

Management

• Monetization of Viral AI Content: Platform creator funds reward high engagement over content authenticity.
• Inadequate Platform Risk Review: Social media firms fail to assess safety risks of AI religious scams.

Technology

• Generative AI Voice Cloning: AI tools easily clone pastor voices from public weekly livestreams.
• Realistic Deepfake Video Tech: Advanced AI video generation makes fake sermons appear highly authentic.

Data Inputs

• Unprotected Public Sermons: Abundant online audio files provide easy training data for voice cloning.
• Lack of AI Content Labeling: Missing labels make it hard for users to identify AI-generated content.

Human Factors

• High Trust in Religious Leaders: Followers naturally trust pastors, making them highly vulnerable to scams.
• Low Digital Literacy of Users: Many churchgoers struggle to identify subtle robotic cues in fake media.
• Predisposition to Delusions: Vulnerable users have religious delusions reinforced by interactive AI.

Process and Methods

• Inadequate Account Verification: Platforms fail to verify official figures, letting fake accounts proliferate.
• Reactive Moderation Systems: Platforms rely on manual reports rather than proactive detection of deepfakes.

Regulatory Environment

• Lack of Deepfake Regulations: Absence of clear laws governing unauthorized AI impersonation of citizens.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides clear, detailed accounts of multiple religious figures being impersonated using AI deepfakes and voice clones, as well as the societal and psychological concerns surrounding religious chatbots. The roles of the AI systems as tools for human scammers are explicitly described.
• Ambiguities identified: The exact financial losses suffered by the victims of these scams are not quantified in the report.

This incident involves scammers using AI deepfakes and voice cloning to impersonate religious leaders in the US and Philippines for financial fraud. While it highlights the growing ease of creating convincing synthetic media and raises concerns about psychological vulnerability to AI, the national security threat remains low, as it does not target critical infrastructure, state sovereignty, or systemic economic stability.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: United States
• Other affected: Philippines
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding the proliferation of deceptive synthetic media rather than an immediate national security crisis.
• Autonomy: Human-controlled. AI systems were utilized as tools by human operators to generate fraudulent media and clone voices.
• Novelty: Evolved capability. Applies existing generative AI and voice cloning technologies to a new target demographic (religious communities) for financial exploitation.

Impact by dimension

• Physical security: Negligible. No physical security threat, kinetic attacks, or critical infrastructure compromise reported in this incident.
• Information security: Minor. Use of deepfakes and synthetic media to impersonate religious figures represents an evolved social manipulation capability, though currently limited to financial fraud rather than state-sponsored geopolitical influence.
• Sovereignty: Negligible. No direct threats to state authority, government decision-making, or core constitutional processes.
• Economic security: Negligible. Financial fraud targeted individual congregants for minor donations, presenting no systemic threat to national economic stability or strategic industries.
• Societal stability: Minor. Potential for localized societal disruption and psychological harm through the reinforcement of delusions via chatbots and erosion of trust in community religious institutions.