Misinformation Reported in TikTok's Search Results Despite Moderation by AI and Human

TikTok's search recommendations reportedly contained misinformation about political topics bypassing both AI and human content moderation.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The TikTok search algorithm generated and spread false or misleading information regarding health, politics, and news events, which can lead to inaccurate beliefs and potential physical harm.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Young users are increasingly relying on TikTok as a primary search engine, trusting it for critical health and news information, which leads to unsafe use when acting on medical misinformation.
• 1.2 Exposure to toxic content: The algorithm exposes users to toxic and harmful content, including dangerous DIY drug recipes and unsafe herbal abortion methods like ingesting pennyroyal.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The search algorithm's recommendation and query suggestion systems unintentionally surfaced and promoted harmful misinformation to users post-deployment.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The search algorithm generates search suggestions and curates content feeds, which requires transparency regarding how results are ranked and suggested to prevent systemic misinformation.

AI system and alleged parties

• AI system: TikTok search algorithm
• AI purpose: Content Search; Content Recommendation
• Behaviour type: Assistant
• Alleged developer: TikTok
• Alleged deployer: TikTok
• Alleged harmed parties: young TikTok users, TikTok users, Gen Z TikTok users

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Minor, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes, the report explicitly describes the spread of toxic and harmful content, including dangerous medical recipes and conspiracy theories.

Directly caused: The TikTok search algorithm directly surfaced and suggested search terms for toxic content, such as DIY hydroxychloroquine recipes, herbal abortion methods, and conspiracy theories, to its massive user base.

Inferred additional harm: Given TikTok's active user base of over a billion people, it is likely that millions of users were exposed to toxic and harmful content surfaced by the algorithm.

Democracy

Reported: Yes, the report describes the spread of election misinformation and conspiracy theories.

Directly caused: The algorithm directly surfaced videos claiming the 2020 election was stolen and that the FBI staged the January 6 Capitol riot, which erodes trust in democratic institutions.

Inferred additional harm: The widespread dissemination of election fraud conspiracies to millions of young voters likely contributes to broader polarization and systemic erosion of democratic norms.

Epistemic

Reported: Yes, the report explicitly describes extensive epistemic harm through the widespread dissemination of misinformation and conspiracy theories.

Directly caused: NewsGuard found that 19.4% of search results for prominent news topics contained misinformation, directly polluting the information ecosystem for users searching for news.

Indirectly caused: The algorithm's auto-suggestions (e.g., 'climate change debunked') steer users toward false beliefs, eroding a shared consensus reality.

Inferred additional harm: As TikTok increasingly replaces Google as a search engine for Gen Z, the systematic promotion of false claims likely causes widespread, long-term epistemic degradation among young populations.

People affected

• Occurrences reported: 1
• People reportedly exposed: 1000000

Potential causes

Management

• Failure to Address Algorithmic Bias: Management failed to adjust algorithms that promote polarizing misinformation.
• Lack of Transparency in Operations: Management refused to answer inquiries on how search algorithms feed results.

Technology

• Flawed Search Suggestion Algorithm: Algorithm suggests highly misleading and conspiratorial search queries.
• Inadequate AI Moderation Detection: AI-driven review failed to detect or flag dangerous medical misinformation.
• Engagement-Biased Search Algorithm: Search algorithm prioritized polarizing and sensational content over accuracy.

Data Inputs

• Lack of Algorithmic Input Disclosure: The political and data inputs driving the algorithm are not disclosed.
• Circumvention via Coded Language: Users bypass word-search filters by avoiding specific terms like 'COVID'.

Human Factors

• User Trust in Flawed Search Engine: Teens increasingly use the platform as a primary search tool, trusting results.
• Uploader Manipulation of Filters: Creators intentionally design videos to evade automated moderation filters.

Process and Methods

• Ineffective Human Moderation Follow-up: Flagged or reported content remained accessible despite violating guidelines.
• Weak Algorithmic Safeguards: Platform lacked automated mechanisms to prevent promotion of dangerous remedies.

Regulatory Environment

• Lack of Algorithm Transparency Rules: No regulatory oversight forcing disclosure of algorithm inputs and biases.

Information quality

• Classification confidence: High
• Reason for confidence: The reports are highly detailed, based on a structured empirical study by NewsGuard with clear methodology (540 search results analyzed across 27 terms). The findings are consistent across both reports, and the role of TikTok's search algorithm is explicitly documented.
• Ambiguities identified: The exact internal mechanics of TikTok's search ranking and suggestion algorithms are proprietary and not disclosed, which limits complete technical understanding of why certain misinformation is prioritized.
• Alternative interpretations: One could argue the primary harm is caused by the human creators of the misinformation videos rather than the AI algorithm itself, though the algorithm's active role in suggesting and ranking these videos makes it an AI safety issue.

TikTok's search algorithm was found to systematically surface misinformation and conspiracy theories to users. While not a direct kinetic threat, the platform's role as a primary information source for young populations creates substantial vulnerability to information warfare and threatens societal stability through the erosion of shared factual reality.

• Overall national security impact: Substantial
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The algorithmic promotion of misinformation represents an ongoing strategic concern rather than an immediate crisis requiring urgent response.
• Autonomy: Full autonomy. The search and recommendation algorithm dynamically curates and ranks content without human intervention in real time.
• Novelty: Evolved capability. Represents an evolved threat where search algorithms, increasingly replacing traditional search engines for youth, actively steer users toward polarized content.

Impact by dimension

• Physical security: Negligible. No direct physical attacks or critical infrastructure disruptions were reported in connection with this incident.
• Information security: Substantial. The algorithm systematically promotes political conspiracy theories and misinformation regarding major events like the Bucha massacre and January 6, undermining the integrity of the public information ecosystem.
• Sovereignty: Minor. Surfacing election fraud and Capitol riot conspiracy theories poses a minor threat to democratic processes and public trust in government institutions.
• Economic security: Negligible. No significant economic manipulation or strategic technology theft was indicated in the report.
• Societal stability: Substantial. Widespread dissemination of conspiracy theories and dangerous health advice to millions of young users erodes social cohesion and threatens public safety at scale.