Facebook’s On-This-Day Feature Mistakenly Showed Painful Memories to Users

Facebook’s “On This Day” algorithm which highlighted past posts on a user’s private page or News Feed confronted unwanted and painful personal memories to its users.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The algorithms failed to perform reliably under varying emotional contexts, exposing users to distressing content due to a lack of capability in sentiment analysis and contextual understanding.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The emotional distress was caused by the automated decisions of Facebook's curation algorithms, which produced unexpected negative outcomes while pursuing the goal of user engagement.

EU AI Act risk tier

• Risk tier: 4 Minimal or No Risk

Minimal or No Risk: The system is a social media content curation and recommendation algorithm used for entertainment and engagement, which generally poses low systemic risk under the EU AI Act.

AI system and alleged parties

• AI system: On This Day, Year in Review (Meta)
• AI purpose: Automated Content Curation; Content Recommendation
• Behaviour type: Autonomous
• Alleged developer: Facebook
• Alleged deployer: Facebook
• Alleged harmed parties: Facebook users having posts about painful events, Facebook users

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Psychological

Reported: The report explicitly describes severe emotional distress, shock, and re-traumatization experienced by users when confronted with painful memories.

Directly caused: At least 5 specific users experienced significant psychological distress, including shock and grief, from being forcefully reminded of suicides, deceased children, fires, and comas.

Inferred additional harm: Given Facebook's massive user base, it is highly likely that thousands of other users experienced similar emotional distress and psychological harm from these features.

People affected

• Occurrences reported: 1
• People reportedly harmed: 5
• People reportedly exposed: 5

Potential causes

Management

• Prioritizing Nostalgia Engagement: Management pushed nostalgia features to drive user engagement.
• Inadequate Risk Mitigation: Features deployed without fully assessing emotional harm risks.

Technology

• Inability to Discern Sentiment: Algorithms cannot read emotional sentiment or context of posts.
• Engagement-Driven Sorting: Algorithms surface highly engaged posts, which are often tragic.
• Inflexible Filtering Rules: Filters only capture limited, explicitly marked life changes.

Data Inputs

• Lack of Machine-Readable Loss Data: Users must document loss publicly for automated filters to work.
• Unlabeled Tragic Content: Posts about tragedies lack explicit metadata indicating sadness.
• Lack of Keyword Filtering: No automatic keyword filters for words like 'RIP' were active.

Human Factors

• Unpredictable Emotional Response: Nostalgia is highly subjective and can trigger painful memories.
• Public Persona Discrepancy: Curated happy public profiles hide complex internal emotions.

Process and Methods

• Opt-Out Design Default: Feature is on by default with no way to completely opt out.
• No Sentiment Analysis in Design: Processes lacked sentiment verification before surfacing memories.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides clear, first-hand accounts of users affected by the algorithm's failures, along with expert commentary on the technical limitations of sentiment analysis. The causal link between the AI's curation decisions and the resulting psychological distress is explicitly documented.

Facebook's 'On This Day' and 'Year in Review' algorithms unintentionally resurfaced painful memories, causing emotional distress to users. While highlighting the limitations of sentiment analysis in consumer AI, the incident has negligible national security implications across all assessed categories.

• Overall national security impact: Negligible
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Unknown
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The incident represents a post-deployment product design issue rather than an active national security crisis.
• Autonomy: Full autonomy. The algorithms operated independently to select and display historical posts to users without real-time human intervention or oversight.
• Novelty: Evolved capability. The incident represents an early, large-scale deployment of automated personalized curation algorithms showing unexpected emotional impacts.

Impact by dimension

• Physical security: Negligible. The incident involves a social media content curation feature and has no impact on physical systems, critical infrastructure, or human safety.
• Information security: Negligible. This was an unintentional product feature failure, not a coordinated information warfare campaign, state-sponsored disinformation, or intelligence compromise.
• Sovereignty: Negligible. The incident does not affect state authority, territorial control, or core government operations.
• Economic security: Negligible. There are no indications of threats to financial systems, strategic technology theft, or national economic stability.
• Societal stability: Negligible. While the feature caused personal emotional distress to individual users, it does not represent mass surveillance, systematic discrimination, or a threat to national societal stability.