Amazon’s Search and Recommendation Algorithms Found by Auditors to Have Boosted Products That Contained Vaccine Misinformation

Evidence of the "filter-bubble effect" were found by vaccine-misinformation researchers in Amazon's recommendations, where its algorithms presented users who performed actions on misinformative products with more misinfomative products.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

Amazon's search and recommendation algorithms spread false and misleading vaccine information by ranking misinformative products highly and creating personalized filter bubbles.

Additional risk subdomains

• 3.2 Pollution of information ecosystem and loss of consensus reality: The algorithms created personalized filter bubbles, progressively showing users more misinformation based on their history, contributing to the degradation of the information ecosystem.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The amplification of vaccine misinformation was driven by the post-deployment operations of Amazon's search and recommendation algorithms, which unintentionally prioritized engaging but misinformative content.

EU AI Act risk tier

• Risk tier: 4 Minimal or No Risk

Minimal or No Risk: The system is an e-commerce search and recommendation algorithm, which generally poses minimal risk to users and does not fall under the high-risk or prohibited categories of the EU AI Act.

AI system and alleged parties

• AI system: None named
• AI purpose: Product Recommendation; Content Search
• Behaviour type: Assistant
• Alleged developer: Amazon
• Alleged deployer: Amazon
• Alleged harmed parties: Amazon Customers

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Substantial, indirect Substantial
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes

Directly caused: Amazon's search and recommendation algorithms directly spread and prioritized misinformative health products, with 10.47% of vaccine-related search results promoting misinformation.

Inferred additional harm: The algorithms likely spread similar health-related misinformation across other medical topics not covered in the 48 audited queries.

Epistemic

Reported: Yes

Directly caused: Amazon's algorithms ranked misinformative search results higher than debunking results and created a filter-bubble effect that progressively contaminated user recommendations with more misinformation.

Indirectly caused: The amplification of anti-vaccine content contributes to broader vaccine hesitancy and the erosion of public trust in medical science.

Inferred additional harm: The widespread exposure to health misinformation likely led to a larger portion of the population adopting false beliefs regarding vaccine safety and efficacy.

People affected

• Occurrences reported: 1

Potential causes

Management

• Engagement Prioritization: Management prioritizes sales metrics over information safety.
• Lack of Health Product Regulation: Failure of Amazon leadership to regulate health-related listings.
• Inadequate Risk Assessment: Management failed to assess risks of vaccine misinformation on platform.

Technology

• Flawed Recommendation Algorithm: Algorithms prioritize engagement over credibility of health products.
• Misinformation Ranking Bias: Search algorithms rank misinformative results higher than debunking ones.
• Filter Bubble Perpetuation: Personalization traps users in a loop of increasing misinformation.

Data Inputs

• Lack of Product Credibility Data: Algorithm inputs do not include trustworthiness or medical verification data.
• User History Tracking: Algorithm uses real-world actions like clicks to build harmful history.
• Unverified Product Metadata: Lack of system checks on health-related metadata of listed products.

Human Factors

• User Trust in Search Engines: Users assume search results are vetted and credible sources of health info.
• Anti-Vaccine Movement Engagement: Active user groups engage with misinformation, driving algorithmic metrics.
• User Click Behavior: Users clicking on dubious items triggers recommendation contamination.

Process and Methods

• Inadequate Algorithmic Audits: Absence of internal automated audits to detect health misinformation spread.
• Weak Product Removal Processes: Relying on external media reports to identify and remove fake products.
• No Credibility Filtering Methods: Information retrieval metrics do not evaluate medical accuracy.

Regulatory Environment

• Lack of E-commerce Regulations: No laws hold online retailers to medical accuracy standards.
• Inadequate Regulatory Pressure: Regulators fail to enforce strict policies on online marketplaces.

Information quality

• Classification confidence: High
• Reason for confidence: The reports are based on a peer-reviewed academic audit with clear methodology, quantitative findings (e.g., 10.47% misinformation rate), and explicit descriptions of the algorithmic mechanisms (ranking bias, personalization, filter bubbles). There is little ambiguity about how the system behaved or what the researchers found.
• Ambiguities identified: The exact algorithms used by Amazon are proprietary, so their internal mechanics are treated as a black box audited from the outside.
• Alternative interpretations: One could argue that the primary issue is the third-party sellers uploading misinformative products rather than the AI itself, but the audit specifically isolates the algorithmic amplification and ranking bias as key drivers of exposure.

An academic audit revealed that Amazon's search and recommendation algorithms unintentionally prioritized and amplified vaccine misinformation, creating filter bubbles for users. While this poses public health and information quality challenges, it represents a minor national security threat as it lacks kinetic impact, state-sponsored coordination, or direct threats to sovereign stability.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Global public
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The issue represents an ongoing, systemic algorithmic challenge rather than an immediate national security crisis requiring urgent response.
• Autonomy: Human-supervised. While the search and recommendation algorithms dynamically generate results, they operate under the ultimate oversight and policy control of Amazon's engineers.
• Novelty: Established threat. Algorithmic bias and recommendation engines creating filter bubbles for misinformation are well-documented and established challenges in the tech industry.

Impact by dimension

• Physical security: Negligible. No direct physical threats, kinetic attacks, or critical infrastructure compromises occurred. Public health concerns from vaccine hesitancy represent indirect safety issues rather than direct physical security threats.
• Information security: Minor. The algorithm amplified misinformation, but this was an unintentional post-deployment ranking bias rather than a coordinated state-sponsored information warfare campaign.
• Sovereignty: Negligible. No impact on state authority, electoral processes, or core government decision-making functions was identified.
• Economic security: Negligible. No threats to strategic industries, financial systems, or national technological competitive advantage were reported.
• Societal stability: Minor. The creation of algorithmic filter bubbles and amplification of vaccine misinformation could contribute to public health skepticism, but does not actively threaten overall societal stability.