Google Photo Merge Decapitates Subject

Google Photos' AI Assistant created a strange hybrid photograph when merging three different pictures from a ski trip.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The AI system failed to perform reliably under varying conditions (differing perspectives of the photos), resulting in a distorted image output due to a lack of robustness in its stitching algorithm.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The glitchy panorama was generated automatically by the Google Photos Assistant AI post-deployment, representing an unintentional software error.

EU AI Act risk tier

• Risk tier: 4 Minimal or No Risk

Minimal or No Risk: The system is an AI used in entertainment or simple photo organization applications, posing low or negligible risks to users and society.

AI system and alleged parties

• AI system: Google Photos' AI Assistant (Google)
• AI purpose: Automated Content Curation; Image Retouching
• Behaviour type: Assistant
• Alleged developer: Google
• Alleged deployer: Google
• Alleged harmed parties: Alex Harker

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 1
• People reportedly exposed: 1

Potential causes

Management

• Prioritizing Novelty: Focus on pushing automated features without rigorous edge-case testing.

Technology

• Flawed Stitching Algorithm: The algorithm incorrectly combined photos with changing subjects.
• Poor Depth Perception: System failed to understand spatial layering of trees and people.

Data Inputs

• Inconsistent Photo Sequence: Input photos had mismatched subjects across the sequence.

Human Factors

• Rapid Photo Capture: User captured overlapping scenes with and without friends.

Process and Methods

• Unchecked Automation: System generated and suggested content without quality checks.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly describes a benign, humorous software glitch with no negative consequences, making the classification straightforward.

A benign and humorous consumer software glitch where Google Photos' AI Assistant incorrectly stitched panoramic ski trip photos, creating a distorted image that went viral on Reddit. The incident has absolutely no national security implications, posing zero threat to physical, informational, sovereign, economic, or societal security.

• Overall national security impact: Negligible
• Scope: Unknown
• Primary target: No clear primary
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. This event poses no active or developing threat to national security, representing a minor software performance issue.
• Autonomy: Human-supervised. The AI assistant automatically generated the panorama, but the final output was managed and shared voluntarily by the user.
• Novelty: Established threat. Image stitching errors and algorithmic glitches in consumer photo editing software are well-known and common occurrences.

Impact by dimension

• Physical security: Negligible. The incident is a benign consumer software glitch with no impact on physical systems, infrastructure, or human safety.
• Information security: Negligible. No intelligence compromise, disinformation campaigns, or classified data exposure occurred. It was a humorous photo stitching error.
• Sovereignty: Negligible. There was no impact on state authority, government operations, electoral systems, or constitutional processes.
• Economic security: Negligible. No financial systems, strategic technologies, or economic stability were threatened by this consumer-level software error.
• Societal stability: Negligible. No threat to societal stability, civil liberties, or human rights. The incident was viewed as a lighthearted and viral social media post.