South Korean Actor Kim Seon-ho's Likeness Allegedly Misused in Purported Deepfake Impersonation Attempts Demanding Money

Fantagio Entertainment, the agency of actor Kim Seon-ho, reportedly warned of recent impersonation scams and purported deepfake videos allegedly misusing his likeness and demanding money. The agency stated that neither Kim nor his staff would solicit funds or personal information, and pledged legal action.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves malicious actors using deepfake technology to impersonate celebrities and scam fans for financial gain.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was caused by human malicious actors intentionally using deployed deepfake generation tools to scam fans.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the creation and distribution of deepfakes, which are subject to transparency obligations under the EU AI Act to ensure users are aware they are interacting with AI-generated content.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Deepfake Video Generation; Social Media Content Generation
• Behaviour type: Tool
• Alleged developer: Unknown voice cloning technology developers, Unknown deepfake technology developers
• Alleged deployer: Unknown scammers impersonating Kim Seon-ho, Unknown scammers
• Alleged harmed parties: Kim Seon-ho, General public of South Korea, Fans of Kim Seon-ho

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the creation and spread of deepfake videos and gambling advertisements.

Directly caused: Malicious actors created and distributed deepfake videos and online gambling ads impersonating celebrities.

Inferred additional harm: It is likely that these deepfakes were widely circulated across various social media platforms, exposing thousands of fans to deceptive and malicious content.

Psychological

Reported: The report explicitly describes psychological distress experienced by the targeted celebrities.

Directly caused: The five targeted celebrities experienced distress and anxiety due to the unauthorized exploitation of their likenesses.

Inferred additional harm: Fans who were scammed out of money likely experienced significant distress and anxiety, potentially affecting dozens of individuals.

Epistemic

Reported: The report explicitly describes the spread of false information through the misuse of deepfakes.

Directly caused: The creation of deepfake videos fabricated false scenarios where celebrities were soliciting money or endorsing gambling.

Inferred additional harm: These deepfakes erode trust in online media and celebrity communications, misleading fans into believing false representations.

People affected

• Occurrences reported: 4
• People reportedly harmed: 5
• People reportedly exposed: 5

Potential causes

Management

• Reactive Agency Protection: Agencies lack proactive defense mechanisms to protect stars' likenesses.

Technology

• Generative AI Deepfake Tools: Advanced AI tools allow realistic replication of actor likenesses.
• Lack of Deepfake Detection: Platforms lack automated systems to detect and block deepfakes.

Data Inputs

• Publicly Available Media: Abundant online photos and videos of stars used to train deepfakes.

Human Factors

• Credulous Social Media Users: Fans easily fall for financial scams believing they are talking to stars.
• Malicious Intent of Scammers: Scammers exploit trust in celebrities for financial gain.

Process and Methods

• Reactive Monitoring Systems: Agencies rely on manual monitoring after the deepfakes are spread.
• Ineffective Takedown Methods: Slow removal processes allow deepfakes to remain online longer.

Regulatory Environment

• Inadequate Legal Deterrents: Current laws fail to prevent illegal deepfake production and distribution.

Information quality

• Classification confidence: Medium
• Reason for confidence: The report clearly identifies the nature of the threat (deepfake impersonation scams) and the affected parties (several South Korean celebrities and their agencies). However, it lacks specific technical details about the AI tools used, the exact number of victims, and the total financial losses incurred.
• Ambiguities identified: The specific AI models used to generate the deepfakes are not identified, and the exact scale of the financial fraud is not quantified.
• Alternative interpretations: None. The events are clearly described as malicious deepfake impersonation scams.
• Missing information: Specific names of the AI generation tools or platforms used, the number of fans who fell victim to the scams, and the total monetary value of the financial losses.

The incident involves malicious actors using AI-generated deepfakes of South Korean celebrities to conduct financial impersonation scams. While causing distress and financial harm to individuals, it represents a standard cybercrime threat with negligible direct impact on national security, critical infrastructure, or state sovereignty.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: South Korea
• Other affected: Unknown
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing, persistent threat of consumer fraud rather than an immediate national security crisis.
• Autonomy: Human-controlled. The AI system is used strictly as a tool by human operators to generate content, with no autonomous decision-making.
• Novelty: Established threat. Celebrity deepfakes and online impersonation scams are well-documented and frequently occurring methods of cybercrime.

Impact by dimension

• Physical security: Negligible. No physical security threat, kinetic attacks, or critical infrastructure compromise indicated in the reports.
• Information security: Minor. Involves synthetic media and deepfake generation, but targeting is limited to celebrity impersonation for financial fraud rather than state-level disinformation.
• Sovereignty: Negligible. No impact on government decision-making, electoral systems, or sovereign state authority.
• Economic security: Minor. Financial fraud and scams targeting individual fans, with no systemic risk to national financial infrastructure or strategic technological assets.
• Societal stability: Minor. Causes localized distress and financial harm to targeted individuals and fans, but does not threaten overall societal stability or constitute systematic human rights violations.