AI-Generated Profiles Used in Disinformation Campaign Targeting Ukrainians

According to security reports by Meta, fictitious personas with GAN-generated profile pictures were used by people operating in Russia and Ukraine to push a disinformation campaign targeting Ukrainian social media users, and were taken down.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale

The incident involved the coordinated use of AI-generated profile pictures to run a state-aligned disinformation and influence campaign targeting the Ukrainian public during a military invasion.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The campaign was intentionally orchestrated by human threat actors who utilized deployed GAN technology to generate realistic profile pictures for fake personas.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the use of AI-generated profile pictures to create fake personas, which falls under 'AI-generated content such as deepfakes' requiring transparency under the EU AI Act.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Deepfake Image Generation; Social Media Content Generation
• Behaviour type: Tool
• Alleged developer: unknown
• Alleged deployer: media organizations in Crimea, individuals in the Donbass region, individuals in Russia
• Alleged harmed parties: Ukrainian social media users

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Minor, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes, the report describes the creation and spread of malicious propaganda and disinformation using fake personas.

Directly caused: The network used AI-generated profile pictures to establish fake personas that published claims about the West betraying Ukraine and Ukraine being a failed state.

Inferred additional harm: It is likely that similar AI-generated profile pictures were used in other undetected influence operations on platforms like Twitter, Telegram, and VK, exposing thousands more users to malicious propaganda.

Democracy

Reported: Yes, the report describes an attempt to disrupt public conversation and undermine trust in the Ukrainian government during a military conflict.

Directly caused: The influence campaign used fake personas to publish claims discrediting Ukraine as a failed state, aiming to manipulate public opinion.

Inferred additional harm: The use of synthetic media to run coordinated inauthentic behavior campaigns during active warfare threatens democratic sovereignty and the integrity of public discourse at a national scale.

Epistemic

Reported: Yes, the report describes the propagation of false claims and disinformation through fake independent news outlets.

Directly caused: The fake personas ran websites masquerading as independent news outlets, publishing false claims that the West betrayed Ukraine and that Ukraine is a failed state.

Inferred additional harm: The proliferation of highly realistic fake personas and news sites contributes to the broader erosion of shared truth and trust in legitimate journalism.

People affected

• Occurrences reported: 1
• People reportedly exposed: 4500

Potential causes

Management

• Underinvestment in AI Safety: Inadequate resources allocated to proactively detect synthetic media accounts.

Technology

• GAN-Generated Profile Pictures: GAN technology allowed threat actors to create realistic, fake profile photos.
• Inadequate AI Detection Tools: Platforms struggled to automatically flag GAN-generated profile images.

Data Inputs

• Lack of Image Provenance Data: Missing metadata made it hard to verify if profile images were AI-generated.
• Unlabeled Synthetic Media: No indicators existed to distinguish GAN photos from real human portraits.

Human Factors

• User Susceptibility to Fake Profiles: Users trusted realistic GAN-generated avatars, aiding social engineering.
• Cognitive Bias Toward Realism: People assume high-quality profile photos belong to real individuals.

Process and Methods

• Weak Identity Verification: Account creation processes did not verify the real identity behind GAN photos.
• Delayed Detection of Fake Networks: Manual investigation was required to link GAN accounts to influence networks.

Regulatory Environment

• Lack of Deepfake Regulations: No legal frameworks mandated disclosure or labeling of GAN-generated media.

Information quality

• Classification confidence: High
• Reason for confidence: The reports from Meta and major news outlets clearly outline the role of GANs in generating profile pictures for the coordinated inauthentic behavior network. The scale of the network and its followers are explicitly quantified, and the distinction between the AI-assisted persona creation and the separate Ghostwriter hacking campaign is well-defined.
• Ambiguities identified: The exact GAN model used to generate the profile pictures is not specified, and the exact number of views or impressions the campaign received across all platforms (like Telegram or VK) remains unquantified.
• Alternative interpretations: The hacking attempts by Ghostwriter could be conflated with the AI-driven influence campaign, but the reports treat them as separate, parallel operations.

Pro-Russian actors utilized GAN-generated profile pictures to create highly realistic fake personas for a coordinated disinformation campaign targeting Ukraine during the 2022 invasion. While the campaign's direct reach was limited, it represents a substantial information warfare effort to undermine state sovereignty during active kinetic conflict.

• Overall national security impact: Substantial
• Scope: Multiple nations
• Primary target: Ukraine
• Other affected: Russia
• Alleged perpetrator: Pro-Russian actors

Threat characteristics

• Imminence: Immediate. The campaign occurred during the active, fast-moving initial phase of the Russian invasion of Ukraine, requiring immediate platform intervention.
• Autonomy: Human-controlled. The AI system acted purely as a tool to generate static profile pictures under the direct control and intent of human operators.
• Novelty: Evolved capability. Represents an evolved capability of using generative AI to enhance the credibility of traditional coordinated inauthentic behavior campaigns.

Impact by dimension

• Physical security: Negligible. No physical infrastructure or human safety was directly targeted or compromised by the AI-generated profile pictures.
• Information security: Substantial. State-aligned actors used GAN-generated personas to run fake news outlets and spread disinformation targeting Ukraine during an active military invasion, coinciding with military hacking.
• Sovereignty: Substantial. Foreign actors used AI-generated personas to spread claims that Ukraine was a 'failed state', directly targeting state legitimacy and public trust during a military invasion.
• Economic security: Negligible. No significant economic impact or strategic technological theft was reported in connection with the GAN-generated profile pictures.
• Societal stability: Minor. The campaign aimed to manipulate public opinion and erode social cohesion, but had limited reach with under 4,500 total followers before being dismantled.