Purportedly AI-Manipulated Medical Scam Advertisement Reportedly Used Bulgarian TV Host and Physician's Likenesses

A purportedly AI-manipulated scam advertisement circulating online appeared to use the likenesses and altered media of Bulgarian television host Simeon Ivanov and physician Spas Spaskov to promote an unregistered purported joint-treatment product. They reportedly denied involvement, and the product was described in reporting as ineffective.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The primary incident involves malicious actors using AI-generated deepfakes of public figures to run fraudulent advertisements and scam victims for financial gain.

Additional risk subdomains

• 4.1 Disinformation, surveillance, and influence at scale: The report highlights how deepfakes are being used as weapons of hybrid warfare, referencing fake videos of world leaders like Putin and Zelenskyy to influence public opinion.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The harm was caused by malicious human actors intentionally deploying AI deepfake technology post-deployment to commit financial fraud.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the creation of deepfakes, which under the EU AI Act are subject to transparency obligations to ensure users are informed they are interacting with AI-generated content.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: Unknown voice cloning technology developers, Unknown deepfake technology developers
• Alleged deployer: Unknown scammers
• Alleged harmed parties: Spas Spaskov, Simeon Ivanov, People seeking medical advice, General public of Bulgaria, General public, Epistemic integrity

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Catastrophic. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Catastrophic, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Minor
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: The report describes global investment scams using deepfakes causing over 40 billion euros in profit for criminal groups.

Directly caused: The average financial loss per occurrence is not specified, but the total reported losses from related deepfake investment scams exceed 40 billion euros.

Inferred additional harm: Substantial additional financial losses were likely incurred by Bulgarian citizens purchasing the fraudulent medicine, though the exact amount is unquantified.

Malicious content

Reported: The report explicitly describes malicious deepfake videos and advertisements created to deceive the public.

Directly caused: Deepfake videos of Dr. Spas Spaskov and TV host Simeon Ivanov were created and spread on social media to promote fake medicine.

Inferred additional harm: Numerous other deepfakes, including fake videos of political leaders like Zelenskyy and Putin, were spread widely across social networks.

Democracy

Reported: The report mentions deepfakes being used as weapons of hybrid warfare, specifically referencing fake videos of Zelenskyy and Putin.

Indirectly caused: A deepfake of Vladimir Putin declaring peace temporarily misled a large portion of the public during the Russia-Ukraine war.

Inferred additional harm: The widespread use of political deepfakes threatens to erode trust in democratic institutions, public broadcasts, and official communications.

Epistemic

Reported: The report describes the creation of highly realistic fake images and videos that make it increasingly difficult for the public to distinguish truth from falsehood.

Directly caused: The deepfake video altered the text and context of a real TV broadcast to make it appear as though a doctor was endorsing a fake medicine.

Indirectly caused: Fake wedding photos of Elon Musk and Mark Zuckerberg spread globally within seconds, demonstrating how easily synthetic media can deceive the public.

Inferred additional harm: The proliferation of indistinguishable deepfakes threatens to degrade the shared information ecosystem, making it harder for citizens to trust any video or audio evidence.

People affected

• Occurrences reported: 1
• People reportedly harmed: 2
• People reportedly exposed: 100000

Potential causes

Management

• Passive Platform Moderation: Tech giants rely on AI-vs-AI battles rather than proactive management.

Technology

• Advanced Deepfake Generation: AI tools make it easy to clone voices and faces with high accuracy.
• Lack of Real-time Detection: No effective automated tools exist to block real-time deepfake video streams.
• Asynchronous Eye Movements: AI facial synthesis still struggles with natural, synchronized eye gaze.

Data Inputs

• Sufficient Public Media Data: Abundant public video and audio of celebrities enable training of AI clones.
• Training Duration of Two Weeks: A two-week training period is enough to create nearly indistinguishable clones.

Human Factors

• Lack of Critical Thinking: Viewers and journalists fail to verify the identity of people on camera.
• Overconfidence in Video Calls: Users trust real-time video communication channels blindly.
• Easy Deception of Public: A large part of the public is easily misled by realistic deepfakes.

Process and Methods

• Inadequate Verification Methods: Lack of control questions to verify the identity of interviewees.
• Delayed Content Moderation: Platforms rely on post-facto reporting to take down fraudulent ads.

Regulatory Environment

• Lack of Deepfake Regulation: No regulatory or technical frameworks prevent high-level deepfake creation.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly outlines the specific scam involving the Bulgarian doctor and TV host, and provides expert commentary on the technical ease of creating deepfakes. The primary classifications are well-supported by the text.
• Ambiguities identified: The exact AI tools or platforms used to generate the deepfakes are not specified.
• Alternative interpretations: The incident could also be viewed through the lens of hybrid warfare due to the mentions of Putin and Zelenskyy, but the primary local event reported is a commercial fraud scam.

The incident involves the use of AI-generated deepfakes in Bulgaria to impersonate a doctor and TV host for commercial fraud. While the report references global concerns about hybrid warfare and large-scale scams, the immediate national security impact of this specific incident is minor, representing localized criminal activity rather than a state-level threat.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: Bulgaria
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. This represents an ongoing, strategic threat regarding the proliferation of synthetic media rather than an active national security crisis.
• Autonomy: Human-controlled. The AI acted as a generation tool directly controlled and deployed by human actors to commit fraud.
• Novelty: Established threat. Deepfakes used for financial scams and public impersonation are well-established methods with many historical precedents.

Impact by dimension

• Physical security: Negligible. No kinetic attacks, autonomous weapons deployment, or critical infrastructure manipulation were reported in this incident.
• Information security: Minor. The primary incident is financial fraud, but the report notes the broader context of deepfakes being used as hybrid warfare tools, posing a minor information security threat.
• Sovereignty: Negligible. No state authority, electoral systems, or core government operations were disrupted or targeted in this specific incident.
• Economic security: Minor. The incident involves localized commercial fraud. While global deepfake scams cause significant losses, this specific event does not threaten strategic economic stability.
• Societal stability: Minor. The exploitation of public trust using fake medical endorsements poses minor risks to public health and trust, but does not threaten overall societal stability.