.avif){kind=small}
Social media platforms including TikTok, YouTube, Instagram, and Twitch utilize automated content moderation algorithms that flag specific words without context. This has forced content creators to adopt 'algospeak'—a system of euphemisms and misspellings—to avoid demonetization or content removal. This phenomenon disproportionately affects marginalized groups, such as LGBTQ and BIPOC creators, by silencing their ability to discuss sensitive topics and effectively altering human language in real-time.
TikTok's, YouTube's, Instagram's, and Twitch's use of algorithms to flag certain words devoid of context changed content creators' use of everyday language or discussion about certain topics in fear of their content getting flagged or auto-demonetized by mistake.
The automated moderation systems perform unequally across demographic groups, disproportionately targeting and silencing the legitimate speech of marginalized communities such as LGBTQ and BIPOC creators.
The suppression of legitimate speech and the resulting rise of algospeak are unexpected and unintentional outcomes of deploying automated content moderation systems.
Minimal or No Risk: The automated content moderation filters and recommendation algorithms described are similar to spam filters, which pose low or negligible risks under the EU AI Act and have no additional regulatory obligations.
Severity ratings are based on this scale.
Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.
Reported: The report explicitly describes financial losses due to video demonetization and content throttling, but does not provide a specific dollar figure.
Directly caused: Loss of monetization and ad revenue for creators whose content is demonetized or down-ranked by the algorithms.
Indirectly caused: Reduced audience growth and brand partnership opportunities due to suppressed algorithmic reach.
Inferred additional harm: Cumulative financial losses across the creator economy are likely in the millions of dollars, particularly impacting independent and marginalized creators.
Reported: The report explicitly notes that 'euphemisms are especially common in radicalized or harmful communities' to evade restrictions.
Directly caused: The algorithms' simplistic word-filtering triggers the creation of 'algospeak' by malicious groups, allowing them to bypass safety filters.
Indirectly caused: Continued exposure of users to harmful content, such as pro-anorexia material or anti-vaccine misinformation, that successfully evades filters using code words.
Inferred additional harm: Widespread evasion of safety filters by bad actors, leading to the ongoing proliferation of toxic and harmful content across social media platforms.
Reported: The report explicitly states that automated content moderation 'disproportionately affects the LGBTQIA community and the BIPOC community'.
Directly caused: Marginalized creators have their legitimate content disproportionately flagged, down-ranked, or demonetized compared to non-marginalized creators.
Indirectly caused: Alienation and silencing of marginalized voices, forcing them to use coded language to discuss their identities and experiences.
Inferred additional harm: Systemic suppression of marginalized communities' online presence, reinforcing societal inequalities and reducing their visibility in public discourse.
Reported: The report explicitly describes 'collateral damage of literal speech' and 'silencing marginalized communities', which relates to freedom of expression.
Directly caused: Infringement on creators' freedom of expression as automated systems censor or suppress legitimate speech without context.
Indirectly caused: Marginalized groups face barriers to organizing, discussing oppression, and sharing educational health information.
Inferred additional harm: Large-scale, systemic restriction of free speech and cultural expression across major global communication platforms.
Reported: The report explicitly describes psychological distress and frustration experienced by creators who feel silenced and forced to self-censor.
Directly caused: Creators experience distress, anxiety, and frustration due to arbitrary censorship and the need to constantly 'dull down' their language.
Indirectly caused: Users struggling with mental health may face barriers in finding support or having frank conversations about suicide due to algorithmic suppression of terms like 'suicide'.
Inferred additional harm: Widespread mild to moderate psychological distress, anxiety, and feelings of alienation among marginalized creators who feel systematically silenced.
Reported: The report describes 'algorithms causing human language to reroute around them in real time', altering common vernacular.
Directly caused: Real-time distortion of human language and communication norms, forcing users to adopt a 'brand-safe lexicon' devoid of natural context.
Indirectly caused: Confusion and degradation of public discourse, where serious topics (medical health, suicide, war) are discussed using childish or nonsensical terms.
Inferred additional harm: Long-term linguistic drift and fragmentation of shared reality, where online communities develop highly insular, algorithmically-driven dialects.
Automated content moderation algorithms on major social media platforms have driven the rise of 'algospeak' as users attempt to bypass contextual blindspots. While this disproportionately impacts marginalized creators and allows harmful groups to evade filters, it presents minor national security concerns, primarily limited to long-term digital human rights and minor information monitoring challenges.