Russia-Linked AI CopyCop Site Identified as Modifying and Producing at Least 19,000 Deceptive Reports

In early March 2024, a network named CopyCop began publishing modified news stories using AI, altering content to spread partisan biases and disinformation. These articles, initially from legitimate sources, were manipulated by AI models, possibly developed by OpenAI, to disseminate Russian propaganda. Over 19,000 articles were published, targeting divisive political issues and creating false narratives.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale

The incident involves state-aligned actors using generative AI to conduct a coordinated, large-scale disinformation campaign targeting democratic elections and public opinion.

Additional risk subdomains

• 3.2 Pollution of information ecosystem and loss of consensus reality: The automated generation of over 19,000 articles across multiple fake websites directly contributes to the pollution of the online information ecosystem.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The disinformation campaign was intentionally designed and executed by human operators using deployed generative AI systems to influence public opinion.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the use of AI to generate text and images for fake news websites. Under the EU AI Act, AI systems that generate or manipulate image, audio, or video content (deepfakes) or generate text are classified as Limited Risk, requiring transparency so users are aware they are interacting with AI-generated content.

AI system and alleged parties

• AI system: ChatGPT, DALL-E 3 (OpenAI)
• AI purpose: Social Media Content Generation; Translation
• Behaviour type: Tool
• Alleged developer: OpenAI, ChatGPT
• Alleged deployer: Russia-linked network, CopyCop
• Alleged harmed parties: Journalism, General public, Democracy

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Severe
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Minor, indirect Minor
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Substantial
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the creation and spread of malicious content designed to influence elections and stoke political division.

Directly caused: The CopyCop network directly generated over 19,000 biased and plagiarized articles across 11 websites using generative AI tools.

Indirectly caused: The malicious content was amplified by other Russian networks like Doppelganger and Portal Kombat, spreading across platforms like X, Telegram, and YouTube.

Inferred additional harm: It is likely that millions of social media users were exposed to these AI-generated articles and related posts, though direct engagement was reported as low.

Democracy

Reported: The report explicitly describes attempts to undermine democratic norms, specifically targeting faith in the integrity of the U.S. election system.

Directly caused: The AI-generated content directly targeted political campaigns, candidates, and democratic processes to sow discord and weaken public trust.

Indirectly caused: The operations forced government agencies, researchers, and social media platforms to divert significant resources to monitor, expose, and debunk these campaigns.

Inferred additional harm: The cumulative effect of multiple foreign state-sponsored AI operations could gradually erode public trust in democratic institutions and election outcomes over time.

Epistemic

Reported: The report explicitly describes epistemic harm through the large-scale generation of false, misleading, and plagiarized news articles.

Directly caused: The CopyCop network generated over 19,000 articles that translated, edited, and modified legitimate news to add partisan bias and false narratives.

Indirectly caused: The dissemination of these articles polluted the online information ecosystem, making it harder for users to distinguish between authentic journalism and AI-generated propaganda.

Inferred additional harm: The widespread deployment of automated text generation tools by adversaries likely contributes to a broader erosion of shared reality and trust in mainstream media.

People affected

• Occurrences reported: 1
• People reportedly exposed: 250000

Potential causes

Management

• State-Sponsored Orchestration: Foreign governments funded and directed networks to deploy AI disinformation.

Technology

• AI Voice Cloning: Adversaries cloned the FBI director's voice to fabricate voter fraud claims.
• Generative LLM Plagiarism: LLMs were used to easily plagiarize, translate, and rewrite news content.
• Automated Content Scaling: AI tools enabled the rapid, high-volume generation of biased election stories.

Data Inputs

• Biased Prompt Engineering: System inputs included prompts instructing LLMs to inject political bias.
• Unverified Source Material: Adversaries fed legitimate news into LLMs to modify and weaponize them.
• Lack of Input Safeguards: Commercial AI models lacked filters to detect malicious political manipulation.

Human Factors

• Prompt Leakage Oversight: Operators left system prompts visible in published AI-generated articles.
• Targeting Vulnerable Readers: Operations targeted specific voter demographics to deepen political division.

Process and Methods

• Inadequate Content Moderation: Social media platforms struggled to proactively block automated fake accounts.
• Automated Article Generation: Propagandists automated the rewriting and posting of thousands of articles.

Regulatory Environment

• Weak Hosting KYC Controls: Weak KYC allowed adversaries to easily buy web hosting infrastructure.

Information quality

• Classification confidence: High
• Reason for confidence: The reports from Recorded Future, Clemson University, and the U.S. government provide highly detailed and consistent accounts of the CopyCop network's operations, including specific numbers of articles (19,000), websites (11), and the exact AI models used (ChatGPT, DALL-E 3). The inclusion of leaked prompts directly confirms the role of generative AI.
• Ambiguities identified: The exact level of organic user engagement with the 19,000 articles is difficult to measure precisely, though researchers note it was generally low except for a few viral stories.
• Alternative interpretations: The campaign could be viewed purely as a traditional geopolitical influence operation, but the central role of LLMs in automating and scaling the content generation makes it a clear AI safety/misuse incident.

The CopyCop campaign represents an evolved information warfare capability where Russian-linked actors leveraged commercial LLMs to automate and scale the production of over 19,000 deceptive articles. While the immediate impact on elections and public opinion was limited due to low content quality and swift detection, it demonstrates the growing national security threat of AI-driven foreign influence operations targeting democratic processes.

• Overall national security impact: Substantial
• Scope: Multiple nations
• Primary target: NATO countries
• Alleged perpetrator: Russia

Threat characteristics

• Imminence: Long-term. Represents an ongoing, strategic capability concern regarding automated foreign influence rather than an active, immediate crisis.
• Autonomy: Human-supervised. AI tools were used to automate content rewriting and translation, but human operators provided the initial prompts and directed the overall campaign.
• Novelty: Evolved capability. Represents a significant advancement in the scale and automation of traditional foreign influence operations through generative AI.

Impact by dimension

• Physical security: Negligible. The incident was purely an information operation with no physical or kinetic components, targeting online media rather than physical infrastructure.
• Information security: Substantial. Russian-linked actors used generative AI to automate and scale information warfare, producing over 19,000 deceptive articles to influence elections in the US, UK, and France.
• Sovereignty: Minor. Attempted to influence democratic elections and government policy in several nations, but failed to significantly alter voter preferences or disrupt government functions.
• Economic security: Negligible. No financial systems, strategic technology, or economic infrastructure were targeted or affected by this campaign.
• Societal stability: Minor. Attempted to polarize society by targeting sensitive issues like the war in Ukraine, but low engagement limited the actual impact on social cohesion.