GAN Faces Deployed by The BL's Fake Account Network to Push Pro-Trump Content on Meta Platforms

A large network of pages, groups, and fake accounts having GAN-generated face photos associated with The BL, a US-based media outlet, reportedly bypassed Facebook moderation systems to push "pro-Trump" narratives on its platform and Instagram.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale

The incident involved a highly coordinated, large-scale influence campaign using AI-generated profile photos to manipulate public opinion and political processes.

Additional risk subdomains

• 3.2 Pollution of information ecosystem and loss of consensus reality: The network used hundreds of fake accounts to artificially amplify content, polluting social media feeds and eroding shared consensus reality for millions of users.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The risk and subsequent platform manipulation were caused by the intentional actions of human operators at 'The BL' who deployed GAN-generated images to deceive users.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes an AI system used to generate synthetic profile photos (AI-generated content/deepfakes), which falls under the category of systems requiring transparency obligations so users are aware they are interacting with AI-generated content.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Deepfake Image Generation; Social Media Content Generation
• Behaviour type: Tool
• Alleged developer: unknown
• Alleged deployer: The BL
• Alleged harmed parties: Instagram users, Facebook users

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Severe
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Minor
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Severe
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the spread of polarizing political content, climate change denial, anti-vaccine rhetoric, and racist tropes.

Directly caused: The GAN-generated profile photos did not directly contain toxic text, but they directly enabled fake accounts to masquerade as real Americans to spread this content.

Indirectly caused: The network spread polarizing political content, climate change denial, and racist tropes to up to 55 million users.

Democracy

Reported: The report explicitly describes the erosion of democratic norms through coordinated inauthentic behavior aimed at manipulating public opinion and political processes.

Directly caused: The use of AI-generated profile pictures directly facilitated the deception of users by making fake accounts appear as authentic Americans.

Indirectly caused: The network artificially amplified political narratives to influence public opinion and democratic processes, targeting 55 million users.

Epistemic

Reported: The report explicitly describes epistemic harm through the fabrication of realistic human faces and the artificial amplification of misleading political narratives.

Directly caused: The GANs fabricated realistic human faces of non-existent people to mislead users about the authenticity of the accounts.

Indirectly caused: The network artificially boosted and spread misleading political content, eroding shared consensus reality for millions of users.

People affected

• Occurrences reported: 1
• People reportedly exposed: 55000000

Potential causes

Management

• Prioritizing Engagement over Safety: Facebook prioritized platform engagement over aggressive CIB mitigation.
• Lack of Coordinated Investigation: Facebook failed to take holistic action against the parent media network.
• Inadequate Proactive Monitoring: Platform relied heavily on third-party journalists to uncover fake networks.

Technology

• GAN-Generated Profile Pictures: GANs created highly realistic fake faces to evade reverse-image searches.
• Automated Posting Systems: High-frequency automated posting bypassed manual rate-limiting controls.
• Evasion of Automated Detection: Inauthentic accounts successfully bypassed Facebook's automated filters.

Data Inputs

• Use of Synthesized Face Data: Neural networks trained on real human faces generated non-existent profiles.
• Repurposed Foreign Profile Data: Existing Vietnamese accounts were altered with fake US demographic details.
• Stock Photo Ingestion: Unsplash stock photos were ingested to masquerade fake administrators.

Human Factors

• User Susceptibility to Fake Profiles: Real users engaged with fake profiles due to realistic GAN-generated faces.
• Manual Coordination by Staff: Real BL staff manually set up groups and assigned fake accounts as admins.
• Exploitation of Social Trust: Using names like 'Josephson Magnolia' to build trust with conservative users.

Process and Methods

• Automated Account Creation: Rapid automated registration allowed quick replacement of banned fake accounts.
• Circumvention of Link Bans: Cloning banned sites onto new domains bypassed Facebook's URL filters.
• Group Admin Loophole Exploitation: Assigning real staff as fallback admins preserved groups after profile bans.

Regulatory Environment

• Lack of Platform Policy Enforcement: Facebook's terms of service on inauthentic behavior were poorly enforced.
• Absence of Synthetic Media Laws: No regulatory framework penalized the deployment of GAN-generated personas.
• Weak Penalties for Repeat Violations: The BL continued operating despite 1,929 historical ad policy violations.

Information quality

• Classification confidence: High
• Reason for confidence: The reports from Snopes, Facebook, Ars Technica, and Wired provide highly consistent and detailed accounts of the coordinated inauthentic behavior network, the specific role of GAN-generated profile photos, and the scale of the operation (55 million users reached, 900+ accounts/pages/groups removed). The technical details of the GAN-generated faces and their detection are well-documented.
• Ambiguities identified: None of significance; the relationship between the AI-generated photos and the broader disinformation campaign is clearly laid out.
• Alternative interpretations: None.

In late 2019, a coordinated inauthentic behavior network linked to The BL used GAN-generated profile pictures to bypass platform detection and run a large-scale influence campaign targeting US users. Reaching 55 million followers, this marked the first major deployment of synthetic faces in information operations, highlighting the evolving national security challenge of AI-enabled disinformation.

• Overall national security impact: Substantial
• Scope: Multiple nations
• Primary target: United States
• Alleged perpetrator: The BL and Epoch Media Group

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic concern regarding AI-enabled disinformation rather than an active, immediate crisis requiring urgent response.
• Autonomy: Human-controlled. AI was used as a generation tool under direct human control to create profile pictures, with humans managing the distribution and account operations.
• Novelty: First-of-its-kind. First documented large-scale deployment of GAN-generated synthetic faces in an active social media influence campaign to evade platform detection.

Impact by dimension

• Physical security: Negligible. The incident involved social media profile picture generation and did not target physical systems, critical infrastructure, or cause physical harm.
• Information security: Substantial. Large-scale coordinated inauthentic behavior network reaching 55 million users, utilizing GAN-generated synthetic faces to evade detection during an information operation.
• Sovereignty: Minor. Attempted to manipulate public political discourse and polarization, but did not directly compromise government decision-making systems or electoral infrastructure.
• Economic security: Negligible. No significant economic infrastructure, financial systems, or strategic technological assets were targeted or compromised.
• Societal stability: Minor. Spread polarizing political narratives and conspiracy theories to millions of users, potentially eroding social cohesion, but without causing active civil unrest.