Viral Image of Pope Francis in a Puffer Jacket Revealed to Be AI-Generated

A viral image of Pope Francis wearing a white puffer jacket was a deepfake produced by the photorealistic-image-generator Midjourney.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The incident involved the generation and viral spread of a discrete false image that misled millions of people into believing the Pope wore a designer jacket.

Additional risk subdomains

• 6.3 Economic and cultural devaluation of human effort: The incident highlights the broader risk of synthetic media reproducing and imitating public likenesses, which the Pope warned could affect the foundations of information and journalism.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The risk of widespread public deception occurred post-deployment due to the AI system generating an unexpectedly realistic image of a public figure that fooled millions.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The incident involves an AI system generating synthetic content (a deepfake), which under the EU AI Act is subject to specific transparency obligations to ensure users are informed that the content is AI-generated.

AI system and alleged parties

• AI system: Midjourney 5 (Midjourney)
• AI purpose: Deepfake Image Generation; Image Generation
• Behaviour type: Tool
• Alleged developer: Midjourney
• Alleged deployer: Eliot Higgins
• Alleged harmed parties: Pope Francis

Harm severity

Highest direct severity in any category: Severe. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Severe, indirect Severe
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Psychological

Reported: The report describes online users feeling shocked, confused, and horrified after realizing they had been duped by the fake image.

Directly caused: N/A

Indirectly caused: The realization of being easily deceived by synthetic media caused feelings of disorientation and horror among some social media users.

Inferred additional harm: N/A

Epistemic

Reported: The report explicitly describes epistemic harm, noting that the image successfully deceived millions of viewers and distorted their relationship with reality.

Directly caused: The AI-generated image directly misled millions of social media users, with one tweet alone gaining over 28 million views and convincing many that the fake photo was genuine.

Indirectly caused: The viral spread of the image contributed to a broader erosion of trust in online visual media, making it harder for the public to distinguish between real and synthetic content.

Inferred additional harm: N/A

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 28100000

Potential causes

Management

• Unrestricted Tool Release: AI developers released powerful generation tools without robust safeguards.
• Priority of Innovation over Safety: Rapid deployment of synthetic tools outpaced defensive detection methods.

Technology

• Advanced Image Synthesis: Midjourney v5 generated photorealistic images that bypassed human detection.
• Unreliable Watermarking: Lack of robust watermarking technology allowed fakes to pass as authentic.
• No Specialist Training Required: Generative tools allowed anyone to produce realistic images with text prompts.

Data Inputs

• Lack of Origin Metadata: Metadata-tagging projects were not widely supported to verify image sources.

Human Factors

• Cognitive Biases of Users: Many online users believed the fake image without verifying its authenticity.
• Inability to Detect Artifacts: Subtle visual flaws like hand distortions were overlooked by the public.
• Excitement and disorientation: Rapid spread of innovations outpaced public understanding of synthetic media.

Process and Methods

• Rapid Social Media Virality: Platforms lacked mechanisms to halt the rapid spread of unverified fakes.
• Insufficient Authentication: No standard verification processes existed before images went viral.

Regulatory Environment

• Absence of Binding Treaties: There was a lack of legally binding international treaties to regulate AI.
• Lack of Mandatory Labeling: No regulatory mandates existed to force platforms to label synthetic content.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts of the creation, virality, and public reception of the Pope deepfake image, including direct quotes from the Pope himself acknowledging the incident.

An individual used Midjourney v5 to generate a photorealistic deepfake of Pope Francis in a puffer jacket, which went viral and deceived millions. While illustrating the growing sophistication of synthetic media and epistemic challenges, the incident carries negligible direct national security implications as it was a benign, non-state-sponsored event with no physical or structural damage.

• Overall national security impact: Minor
• Response level: Moderate
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Unknown
• Alleged perpetrator: Individual

Threat characteristics

• Imminence: Long-term. The incident is a resolved viral event rather than an active national security crisis requiring immediate intervention.
• Autonomy: Human-controlled. The AI functioned strictly as a tool generating an image based on direct text prompts provided by a human user.
• Novelty: Evolved capability. The incident demonstrated a significant advancement in the realism and public believability of consumer-accessible generative AI tools.

Impact by dimension

• Physical security: Negligible. The incident involved a digital image of the Pope and did not impact physical infrastructure, weapons, or human physical safety.
• Information security: Minor. While the deepfake went viral and misled millions, it was created by an individual as a benign depiction rather than a coordinated state-sponsored information warfare or intelligence campaign.
• Sovereignty: Negligible. No sovereign state decisions, electoral systems, or core government operations were compromised or disrupted.
• Economic security: Negligible. The incident did not target financial systems, critical supply chains, or lead to the theft of strategic technology.
• Societal stability: Minor. The viral image caused temporary confusion and epistemic disorientation regarding reality, but did not spark civil unrest, mass surveillance, or systemic human rights violations.