Facebook feed algorithms were known by internal research to have harmed people having low digital literacy by exposing them to disturbing content they did not know how to avoid or monitor.
.avif){kind=small}
Internal Facebook research revealed that its engagement-based feed algorithms disproportionately exposed users with low digital literacy to graphic violence, nudity, and scams. Because these users lacked the technical skills to hide or report content, the algorithm misinterpreted their passive scrolling as approval, creating a feedback loop of harmful content. The research identified that these impacts were most severe for vulnerable populations, including older adults, people of color, and those with lower socioeconomic status.
Facebook feed algorithms were known by internal research to have harmed people having low digital literacy by exposing them to disturbing content they did not know how to avoid or monitor.
The primary reported harm is the systematic and repeated exposure of vulnerable users to toxic content, including graphic violence, nudity, and scams, due to algorithmic feedback loops.
The harm was caused by the post-deployment operations of Facebook's News Feed algorithm, which unintentionally created a toxic feedback loop by misinterpreting passive scrolling as user approval.
Unacceptable Risk: The report describes an AI system that systematically exploits the vulnerabilities of users based on age, education, and socioeconomic status, leading to repeated exposure to disturbing and harmful content.
Highest direct severity in any category: Severe. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.
Reported: Yes, the report notes that users who followed coupon and savings pages were inundated with financial scams, though specific loss amounts are not quantified.
Directly caused: N/A
Indirectly caused: Vulnerable users fell victim to financial scams promoted and recommended by the algorithm, resulting in unquantified financial losses.
Inferred additional harm: Given the scale of exposure to financial scams among low-literacy users globally, it is highly likely that thousands of users suffered significant financial losses, potentially totaling millions of dollars.
Reported: Yes, the report explicitly describes the spread of graphic violence, nudity, hate speech, and scams.
Directly caused: The algorithm directly amplified and distributed graphic violence (13.4% more for low-literacy users) and borderline nudity (11.4% more) to vulnerable populations.
Indirectly caused: N/A
Inferred additional harm: Widespread dissemination of toxic, violent, and inappropriate content to hundreds of millions of low-literacy users globally.
Reported: Yes, the report explicitly describes how low-literacy users (who are disproportionately older, people of color, and lower socioeconomic status) were exposed to significantly more harmful content.
Directly caused: The algorithm fed 11.4% more nudity and 13.4% more graphic violence to low-literacy users compared to high-literacy users, creating unequal safety outcomes.
Indirectly caused: N/A
Inferred additional harm: Systemic unequal outcomes where vulnerable demographic groups bear the brunt of algorithmic harms and toxic content exposure due to design choices.
Reported: Yes, the report mentions 'bad-faith attacks at democracy/civility' and political propaganda via foreign clickbait farms.
Directly caused: N/A
Indirectly caused: The algorithm's promotion of divisive content and political propaganda, especially in 'at-risk' countries, eroded democratic norms and civility.
Inferred additional harm: Large-scale erosion of democratic discourse and political stability in multiple countries due to algorithmic amplification of divisive content.
Reported: Yes, the report describes a user feeling uncomfortable that Facebook recommended a friend request to someone from a dating site, revealing her last name without her explicit consent.
Directly caused: The algorithm used cross-platform tracking or contact matching to recommend friend requests, revealing private information (last names).
Indirectly caused: N/A
Inferred additional harm: Widespread unauthorized inference and sharing of private connections and personal data across millions of users.
Reported: Yes, the report explicitly describes vulnerable users experiencing distress and exacerbated hardships from being exposed to graphic violence, bullying, and threatening content.
Directly caused: The algorithm repeatedly showed graphic violence, bullying, and threatening content to vulnerable users, causing them to disconnect and exacerbating existing mental and emotional hardships.
Indirectly caused: N/A
Inferred additional harm: Given that up to one-third of Facebook's massive global user base has low digital literacy, it is highly likely that millions of vulnerable users experienced significant distress, anxiety, or psychological harm from repeated exposure to disturbing content.
Reported: Yes, the report describes the spread of low-quality information, COVID-19 misinformation, hoaxes, and scams.
Directly caused: The algorithm recommended and amplified misinformation to users who mistook virality for trustworthiness.
Indirectly caused: N/A
Inferred additional harm: Widespread erosion of shared truth and belief in false information (e.g., COVID-19 and 5G conspiracies) among millions of low-literacy users globally.
Internal Facebook research revealed that its News Feed recommendation algorithm created toxic feedback loops that disproportionately exposed low-digital-literacy users to graphic violence, scams, and political propaganda. While the societal impact is widespread, the direct national security implications are minor, primarily manifesting as passive amplification of foreign propaganda and erosion of democratic discourse.