TikTok's "For You" Algorithm Directed New Users towards Disinformation about the War in Ukraine

An investigation by NewsGuard into TikTok’s handling of content related to the Russia-Ukraine war showed its “For You” algorithm pushing new users towards false and misleading content about the war within less than an hour of signing up.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.2 Pollution of information ecosystem and loss of consensus reality

The recommendation algorithm systematically polluted the information ecosystem by serving unverified and false war content to users at scale, undermining a shared consensus reality during a major global conflict.

Additional risk subdomains

• 4.1 Disinformation, surveillance, and influence at scale: The platform's recommendation engine was exploited by state-backed actors to conduct large-scale, coordinated disinformation campaigns targeting public opinion.
• 5.1 Overreliance and unsafe use: Users, particularly young demographics, overrelied on TikTok as an authentic and primary news source for critical real-time war updates.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The risk of spreading disinformation was an unexpected and unintended outcome of the recommendation algorithm pursuing its goal of maximizing user engagement.

EU AI Act risk tier

• Risk tier: 1 Unacceptable

Unacceptable Risk: The system is described as using behavioral manipulation on a vulnerable young audience, leading to emotional manipulation and distress.

AI system and alleged parties

• AI system: TikTok For You algorithm (ByteDance)
• AI purpose: Content Recommendation; Content Curation
• Behaviour type: Autonomous
• Alleged developer: TikTok
• Alleged deployer: TikTok
• Alleged harmed parties: TikTok users, TikTok new users

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Severe. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Severe, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Minor, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Severe, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the spread of toxic Kremlin propaganda and fake war footage directly and indirectly by the incident.

Directly caused: TikTok's recommendation algorithm directly served and amplified false claims of 'neo-Nazi junta' and 'bioweapon laboratories' to new users within 40 minutes.

Inferred additional harm: Millions of users were exposed to toxic state-sponsored propaganda and graphic war imagery, accelerating polarization and hostility.

Democracy

Reported: The report explicitly describes the platform being used for coordinated propaganda and censorship, eroding democratic norms.

Directly caused: TikTok's algorithm amplified state-backed Kremlin propaganda, undermining public understanding of a democratic nation's invasion.

Inferred additional harm: The erosion of public trust and manipulation of political discourse at scale during a major geopolitical crisis.

Psychological

Reported: The report explicitly describes users feeling emotionally manipulated by the highly engaging and graphic war videos fed to them.

Indirectly caused: Quoted users reported that the videos they were shown were designed to get them 'riled up' or to 'emotionally manipulate' them.

Inferred additional harm: Widespread anxiety, distress, and emotional manipulation among millions of young users exposed to graphic and fake war footage without context or warnings.

Epistemic

Reported: The report explicitly details the widespread dissemination of false and misleading information about the war.

Directly caused: The algorithm fed unverified and fabricated videos (e.g., video game footage as real combat, old audio over new videos) to users, corrupting their shared reality.

Inferred additional harm: Millions of users developed false beliefs about the war, severely degrading the global information ecosystem's integrity.

People affected

• Occurrences reported: 1
• People reportedly harmed: 2
• People reportedly exposed: 8

Potential causes

Management

• Prioritizing App Growth: Focus on user retention and engagement over robust safety guardrails.
• Delayed Safety Resource Allocation: Safety and security resources were only increased reactively after the crisis.
• Inconsistent Account Verification: Verifying state-backed propaganda actors while allowing them to spread myths.

Technology

• Engagement-driven Algorithm: Prioritizes watch time and pushes users into disinformation rabbit-holes.
• Easy Audio Reuse Feature: Allows old or unrelated audio to be paired with misleading war footage.
• Inherent Video Moderation Difficulty: Video is a highly complex format for automated moderation algorithms.

Data Inputs

• Lack of Content Labeling: Absence of labels distinguishing disinformation from trusted news.
• Unverified Video Submissions: Unauthenticated video game footage uploaded as real-time war media.
• Misleading Audio Soundtracks: Recycled audio tracks from unrelated past events uploaded to new videos.

Human Factors

• User Overtrust of Platform: Young users perceive TikTok content as authentic and fail to fact-check.
• Lack of User Digital Literacy: Difficulty among general users in identifying simulated video footage.
• Coordinated Influence Campaigns: Pro-Kremlin operatives and paid influencers spreading targeted propaganda.

Process and Methods

• Inadequate Content Moderation: Inability to quickly review and remove viral, unverified war footage.
• Absence of Trustworthiness Indicators: Failure to provide source credibility ratings, warnings, or fact-checks.
• Inadequate Search Filtering: Search suggestions include heavy disinformation in top twenty results.

Regulatory Environment

• Russian Fake News Law: Forced suspension of local uploads, altering the balance of content.
• Lack of Global AI Standards: No uniform regulations requiring algorithmic transparency or safety audits.
• Geopolitical State Influence: State-backed media outlets exploiting platforms to push geopolitical agendas.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, consistent, and empirical evidence of the algorithm's behavior from multiple independent sources (NewsGuard, NYT, Fortune, Guardian). The role of the recommendation algorithm is explicitly defined and tested.
• Alternative interpretations: The incident could be viewed as a moderation failure rather than an algorithmic failure, though the algorithm's active promotion of the content makes it an AI safety issue.

Tiktok's recommendation algorithm autonomously amplified state-backed Kremlin propaganda and false war narratives to new users during the Russia-Ukraine conflict. This represents a substantial information warfare concern, demonstrating how engagement-maximizing AI systems can be exploited to pollute the information ecosystem at scale during active geopolitical crises.

• Overall national security impact: Substantial
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Multiple nations
• Alleged perpetrator: Pro-Kremlin actors

Threat characteristics

• Imminence: Long-term. Represents an ongoing capability and strategic concern regarding how recommendation algorithms handle information during geopolitical crises.
• Autonomy: Full autonomy. The recommendation algorithm operates autonomously to curate and serve content in real-time based on engagement metrics without human oversight.
• Novelty: Evolved capability. Represents an evolved capability where engagement-maximization algorithms are exploited to rapidly scale and target war-related disinformation.

Impact by dimension

• Physical security: Negligible. No physical security threats, kinetic attacks, or critical infrastructure compromise reported in the incident details.
• Information security: Substantial. The recommendation engine amplified state-backed Kremlin propaganda and false war narratives at scale, facilitating automated disinformation delivery during an active geopolitical conflict.
• Sovereignty: Minor. While it manipulated public discourse during a foreign war, there is no direct evidence of domestic electoral manipulation or compromise of core government decision-making.
• Economic security: Negligible. No significant economic manipulation, financial system attacks, or strategic technology theft reported.
• Societal stability: Minor. Large-scale exposure of young demographics to graphic and fake war footage caused emotional distress and polarization, though no civil unrest was directly reported.