TikTok's "Suggested Accounts" Algorithm Allegedly Reinforced Racial Bias through Feedback Loops

TikTok's "Suggested Accounts" recommendations allegedly reinforced racial bias despite not basing recommendations on race or creators' profile photo.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.1 Unfair discrimination and misrepresentation

The recommendation algorithm created demographic filter bubbles that systematically marginalized creators of color and underrepresented minorities, leading to unfair outcomes and representation.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The bias in recommendations was caused by the TikTok recommendation algorithm's post-deployment behavior, which unintentionally amplified human biases through collaborative filtering.

EU AI Act risk tier

• Risk tier: 4 Minimal or No Risk

Minimal or No Risk: The TikTok recommendation algorithm is used in an entertainment application to suggest creators to follow, which poses low or negligible direct risk to safety or fundamental rights under the EU AI Act framework.

AI system and alleged parties

• AI system: TikTok recommendation algorithm (ByteDance)
• AI purpose: Content Recommendation; Behavioral Modeling
• Behaviour type: Assistant
• Alleged developer: TikTok
• Alleged deployer: TikTok
• Alleged harmed parties: TikTok users, TikTok content creators

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Substantial
• Civil rights: direct Negligible, indirect Minor
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Differential treatment

Reported: Yes, the report describes how TikTok's recommendation algorithm suggests creators who physically resemble the followed account, potentially leading to racial and demographic sorting.

Directly caused: The algorithm recommended users of the same race, hair color, and physical appearance when a user followed an account, such as suggesting only hijabi creators after following one hijabi creator.

Indirectly caused: This creates a feedback loop where creators of color and underrepresented minorities are systematically excluded from recommendations to broader audiences, making it harder for them to gain followers.

Inferred additional harm: It is highly likely that this algorithmic behavior systematically restricts the reach of minority creators across the entire platform, reinforcing real-world racial and demographic disparities in digital spaces.

Civil rights

Reported: Yes, the report discusses potential racial bias and the marginalization of underrepresented minorities.

Indirectly caused: The algorithm's feedback loop indirectly limits the equal opportunity of minority creators to participate and succeed on the platform on an equal basis.

Inferred additional harm: The systemic suppression of minority creators' content could be inferred as a passive violation of civil rights regarding equal treatment and non-discrimination in commercial digital public squares.

People affected

• Occurrences reported: 1
• People reportedly exposed: 3

Potential causes

Management

• Prioritizing Engagement: Focusing on user retention metrics over mitigating demographic bubbles.
• Historical Content Suppression: Prior management decisions to bury content from marginalized groups.

Technology

• Collaborative Filtering Bias: The algorithm reproduces human biases by recommending what co-followers liked.
• Automatic Image Featurization: The system may extract physical features directly from user profile pictures.
• Feedback Loop Creation: Recommending similar creators traps users in narrow, homogenous bubbles.

Data Inputs

• Biased User Follow History: User behavior data reflects preexisting societal biases and preferences.
• Profile Photo Demographics: Profile pictures contain visible traits like race, gender, and disabilities.

Human Factors

• User Homophily: Users naturally tend to follow creators who look physically similar to them.
• Preexisting Social Biases: User behavior is influenced by societal biases regarding race and appearance.

Process and Methods

• Lack of Algorithmic Auditing: No robust process to audit or replicate bias claims internally or externally.
• Opaque Proprietary Logic: Proprietary algorithms prevent external scrutiny of potential biases.

Regulatory Environment

• No Algorithmic Bias Mandates: Lack of legal frameworks enforcing fairness in social media recommendations.

Information quality

• Classification confidence: High
• Reason for confidence: The incident is well-documented across multiple reputable news outlets reporting on the same primary source (researcher Marc Faddoul). The findings were independently replicated by journalists, and TikTok's official explanation (collaborative filtering) is clearly presented alongside the researcher's hypotheses, providing a balanced and clear view of the technical behavior.
• Ambiguities identified: The exact technical mechanism (whether it is pure collaborative filtering reproducing human bias, or if it includes active computer vision featurization of profile pictures) remains unconfirmed as TikTok denied using facial features.
• Alternative interpretations: The observed homogeneity could be purely a reflection of existing user follow patterns (collaborative filtering) rather than any active demographic profiling by TikTok's AI.

TikTok's recommendation algorithm was found to suggest accounts based on physical and demographic similarity, creating feedback loops that systematically marginalize minority creators. While raising minor concerns regarding algorithmic bias and societal equity, the incident poses negligible direct threats to national security.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Unknown
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The issue represents an ongoing, systemic algorithmic behavior rather than an active crisis requiring immediate intervention.
• Autonomy: Full autonomy. The recommendation engine autonomously generates and serves account suggestions to users without real-time human intervention.
• Novelty: Established threat. Algorithmic bias and feedback loops in collaborative filtering systems are well-known and established issues in machine learning.

Impact by dimension

• Physical security: Negligible. No threat to physical systems, critical infrastructure, or human safety is indicated in the incident.
• Information security: Negligible. The incident involves an unintentional recommendation bias rather than a coordinated information warfare campaign or intelligence compromise.
• Sovereignty: Negligible. No disruption to state authority, electoral processes, or core government operations was reported.
• Economic security: Negligible. While there are minor potential economic impacts on individual creators, there is no threat to national strategic industries or economic stability.
• Societal stability: Minor. The recommendation algorithm's feedback loop reinforces demographic biases and marginalizes minority creators, posing minor concerns for societal equity and representation.