Alleged State-Sponsored Hackers Escalate Purported Phishing Attacks Using Artificial Intelligence

State-sponsored hackers from North Korea, Iran, Russia, and China are reportedly leveraging artificial intelligence to conduct sophisticated phishing and social engineering attacks. They target global defense, cybersecurity, and cryptocurrency sectors, aiming to steal sensitive information and, in the case of North Korea, cryptocurrencies to help fund its illicit nuclear program.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The primary activity described is the targeted exploitation of specific victims (defense, cybersecurity, and crypto professionals) using AI-generated fake recruiter profiles and highly polished phishing messages for personal or state advantage.

Additional risk subdomains

• 4.2 Cyberattacks, weapon development or use, and mass harm: Threat actors also utilized LLMs to assist with basic coding, scripting, and troubleshooting tasks to support malware development and cyberattacks.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was caused by human threat actors intentionally exploiting deployed generative AI systems to enhance their cyber-espionage and phishing operations.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the use of general-purpose chatbots and generative AI models, which fall under transparency obligations for chatbots and synthetic content.

AI system and alleged parties

• AI system: ChatGPT 4 (OpenAI)
• AI purpose: Writing Assistant; Code Generation
• Behaviour type: Assistant
• Alleged developer: Russian government, North Korean government, Iranian government, Chinese government
• Alleged deployer: Russian hackers, North Korean hackers, Iranian hackers, Chinese hackers
• Alleged harmed parties: Individual professionals on LinkedIn, Global defense companies, Cybersecurity firms, Cryptocurrency exchanges

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Minor
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes, the reports describe the creation and dissemination of malicious phishing emails and fake social media profiles used for social engineering.

Directly caused: Threat actors directly used LLMs to generate clean, convincing phishing emails and draft content for social engineering campaigns targeting experts and organizations.

Inferred additional harm: It is likely that thousands of malicious messages and fake profiles were generated and distributed across platforms like LinkedIn, Telegram, and WhatsApp.

Privacy

Reported: Yes, the reports describe hackers using AI to research targets and attempt to trick users into revealing sensitive information.

Indirectly caused: Threat actors used LLMs to research foreign think tanks, defense experts, and high-profile individuals to gather intelligence.

Inferred additional harm: It is likely that sensitive personal and professional data of targeted individuals was gathered and compromised during these reconnaissance and spear-phishing operations.

Epistemic

Reported: Yes, the reports describe hackers using AI to generate deceptive content, fake identities, and fake recruiter profiles to mislead targets.

Directly caused: Hackers used generative AI to create credible-looking recruiter profiles and draft highly convincing messages to build relationships and deceive targets.

Inferred additional harm: The proliferation of highly polished, AI-generated fake profiles on professional networks degrades the overall trust and shared reality of online professional interactions.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 1000

Potential causes

Management

• Prioritizing Speed Over Safety: Tech firms rushed ChatGPT to market, leaving security as an afterthought.
• Selling Tools Instead of Secure LLMs: Firms monetize defensive tools rather than building inherently secure models.

Technology

• Vulnerable LLM Safeguards: Adversaries found ways to bypass safety filters to generate malicious content.
• Dual-Use Capabilities of LLMs: Models help with coding, scripting, and malware development tasks.
• AI-Enhanced Profile Generation: GenAI creates highly credible recruiter profiles and images on social media.

Data Inputs

• Publicly Available Vulnerabilities: AI models leverage open-source data to identify software vulnerabilities.
• Open-Source Target Intelligence: Adversaries feed target profile data into LLMs to tailor phishing campaigns.
• Foreign Language Training Data: Training on natural language helps attackers overcome native language barriers.

Human Factors

• Trust in Credible Social Profiles: Targets trust polished, AI-generated LinkedIn recruiter profiles over months.
• Human Cognitive Limits in Detection: Victims cannot easily distinguish AI-generated messages from real recruiters.
• Social Engineering Susceptibility: Targets are easily lured by personalized, AI-drafted spear-phishing emails.

Process and Methods

• Hasty Public Release of LLMs: AI models were released publicly before robust security audits were completed.
• Security as an Afterthought: System development prioritized rapid deployment over built-in security controls.
• Inadequate Defensive AI Tools: Defensive tools lag behind offensive AI capabilities used by threat actors.

Regulatory Environment

• Lack of Security-by-Design Mandates: No regulatory frameworks required rigorous safety testing prior to LLM release.

Information quality

• Classification confidence: High
• Reason for confidence: The reports from Microsoft, OpenAI, and cybersecurity experts provide consistent, detailed accounts of how state-sponsored hackers are utilizing LLMs. While specific technical details of the prompts are omitted, the overall operational patterns and the nature of the disruption are clearly documented.
• Ambiguities identified: The exact number of successful compromises and the specific AI models used by some of the non-OpenAI groups remain slightly ambiguous.
• Alternative interpretations: The activities could be viewed purely as traditional cyber-espionage where AI is merely an incidental tool, rather than an AI-specific safety failure.

Disruption of state-affiliated cyber-espionage groups from North Korea, Iran, Russia, and China using LLMs to enhance spear-phishing, social engineering, and basic coding. While the AI capabilities used were early-stage, the incident highlights how generative AI lowers technical and language barriers for sophisticated, state-sponsored cyber operations targeting global defense and technology sectors.

• Overall national security impact: Substantial
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: United States, Japan, and global defense sectors
• Alleged perpetrator: State-affiliated actors from North Korea, Iran, Russia, and China

Threat characteristics

• Imminence: Long-term. The immediate threat was mitigated by disabling the accounts, representing an ongoing strategic concern regarding AI-enabled cyber-espionage capabilities.
• Autonomy: Human-controlled. AI systems were used strictly as writing and coding assistants, with human actors making and executing all final decisions.
• Novelty: Evolved capability. Represents a significant incremental advancement where generative AI is integrated into established cyber-espionage and social engineering workflows.

Impact by dimension

• Physical security: Negligible. No physical security breaches, kinetic attacks, or critical infrastructure disruptions were reported as a result of these AI-assisted activities.
• Information security: Substantial. State-affiliated threat actors from Russia, China, Iran, and North Korea utilized LLMs to enhance spear-phishing campaigns, build deceptive recruiter profiles, and conduct reconnaissance against defense and cybersecurity targets.
• Sovereignty: Minor. State-sponsored espionage groups targeted defense and think tank sectors, representing a minor threat to government information security without disrupting core sovereign functions.
• Economic security: Minor. Targeting of a Japanese cryptocurrency exchange engineer and research into sensitive technologies represents minor economic and technological security risks, managed by disrupting the accounts.
• Societal stability: Negligible. No large-scale threats to societal stability, civil liberties, or population safety were reported in connection with this incident.