.avif){kind=small}
Internal Meta documents reveal that the company's automated ad-personalization and targeting systems have been used to facilitate billions of scam impressions daily, generating significant revenue. Despite internal awareness of the scale of fraud, Meta implemented 'revenue guardrails' that limited enforcement actions against profitable scam advertisers. The system's design, which targets ads based on user interests, inadvertently creates feedback loops that expose vulnerable users to repeated fraudulent content, leading to substantial financial losses for users globally.
Internal Meta documents reviewed by Reuters reportedly show that the company's automated ad and targeting systems generated major revenue from fraudulent ads and exposed users to billions of scam impressions daily. Enforcement was limited by internal revenue guardrails. In one example, a Royal Canadian Air Force recruiter's account was hacked and used to run a crypto scam that defrauded several colleagues after repeated reports allegedly went unanswered.
The incident involves the widespread facilitation of financial scams, crypto fraud, and impersonation schemes targeting users on Meta's platforms, leveraged through automated systems.
The risk is driven by the post-deployment actions of Meta's automated ad-personalization system, which unintentionally creates feedback loops that amplify scam exposure for vulnerable users.
Unacceptable Risk: The system is described as an 'exploitative system targeting vulnerabilities' and 'AI used for subliminal manipulation or behavioral influence leading to harm,' specifically targeting vulnerable users who click on scams with feedback loops that flood their feeds with more fraudulent content.
Severity ratings are based on this scale.
Highest direct severity in any category: Severe. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.
Reported: The report explicitly describes physical harm to people caused indirectly by the incident.
Indirectly caused: The report states that sextortion scams on the platform have led to teenagers committing suicide.
Inferred additional harm: It is likely that other individuals suffered physical self-harm or health issues due to stress or banned medical products, but specific numbers are not detailed.
Reported: The report describes 1 ongoing systemic occurrence causing billions of dollars in global losses, with specific reported losses of C$65,000 for a group of military colleagues and $250,000 in scam crypto ads. The average financial loss per occurrence is estimated in the billions of dollars globally.
Indirectly caused: One victim lost C$40,000 (approx. $28,000 USD), and a group of military colleagues lost a total of C$65,000.
Inferred additional harm: Given that Meta's platforms are involved in a third of US scams and billions of scam ads are shown daily, the total global financial loss to users is inferred to be in the billions of dollars annually.
Reported: The report explicitly describes toxic or malicious content created or spread directly or indirectly by the incident.
Directly caused: Meta's ad-personalization system directly served and amplified 15 billion higher-risk scam advertisements daily to users.
Indirectly caused: Scammers generated and distributed billions of organic scam messages and fraudulent posts.
Inferred additional harm: Millions of additional malicious ads and organic scam posts continue to circulate widely across Meta's platforms daily.
Reported: The report explicitly describes differential treatment of individuals or groups caused directly by the incident.
Directly caused: Meta's ad-personalization system targeted ads based on user interests, meaning users who clicked on a scam were automatically flooded with more scams, disproportionately targeting vulnerable users.
Inferred additional harm: Systemic exploitation of vulnerable demographics (e.g., teenagers, elderly, or less tech-savvy users) who are repeatedly targeted by automated systems.
Reported: The report explicitly describes violations of human or civil rights caused indirectly by the incident.
Indirectly caused: The systematic targeting and exploitation of teenagers through sextortion and users through massive fraud violates their rights to safety and protection from exploitation.
Inferred additional harm: Widespread violations of personal security and consumer rights across millions of affected users globally.
Reported: The report explicitly describes privacy violations caused directly and indirectly by the incident.
Directly caused: Meta's ad-targeting system utilized nonconsensual user dossiers to allow scammers to target vulnerable individuals.
Indirectly caused: Scammers hacked personal accounts (like the Canadian recruiter's) to impersonate them and target their contacts.
Inferred additional harm: Millions of users have had their personal data and behavioral profiles exploited by malicious actors through Meta's targeting tools.
Reported: The report explicitly describes psychological or mental-health harm to people caused indirectly by the incident.
Indirectly caused: Teenagers experienced severe psychological distress from sextortion scams, and victims like the Canadian recruiter experienced distress ('she cried') and military colleagues felt violated.
Inferred additional harm: Widespread anxiety, depression, and trauma among the millions of scam victims who lost their life savings or were blackmailed.
Reported: The report explicitly describes epistemic harm caused directly and indirectly by the incident.
Directly caused: Meta's ad system served ads falsely claiming to represent consumer brands, public figures like Canada's prime minister, or fake job listings.
Indirectly caused: Scammers created fake profiles impersonating military personnel and trusted friends to deceive users.
Inferred additional harm: Widespread erosion of trust in online communications and the authenticity of digital identities on social media platforms.
Reported: The report explicitly describes a CSEA incident involving sextortion scams targeting teenagers, where scammers obtain sexual images under false pretenses and blackmail them.
Indirectly caused: Scammers used the platform to conduct sextortion scams targeting teenagers, obtaining sexual images under false pretenses.
Inferred additional harm: Widespread unreported cases of online grooming and sexual exploitation of minors facilitated by the platform's messaging and targeting systems.
Meta's automated ad-personalization systems have systematically facilitated billions of scam impressions daily, creating severe economic and societal harms globally. While primarily a financial and societal threat, the compromise of military personnel accounts highlights potential vectors for targeted information operations.