A fraudulent Facebook page used deepfake technology to create a video of Croatian immunologist Stipan Jonjić endorsing an anti-parasite product. The video, which featured manipulated audio and visuals, successfully deceived at least one consumer into purchasing the product. The immunologist confirmed the video was a deepfake and that he had no involvement in the promotion.
A Facebook page reportedly posted a video that purported to show Croatian immunologist Stipan Jonjić endorsing an anti-parasite product and recommending it over conventional treatment. The clip was reported to use AI-generated or manipulated media, and Jonjić denied involvement. A woman was reported to have purchased the product after seeing the ad.
Risk classification
- Primary risk domain: 4 Malicious actors
- Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation
The incident involves the unauthorized use of a public figure's likeness and voice in an AI-generated deepfake to perpetrate a financial scam and deceive consumers into buying a fake product.
Causal factors
- Entity: Human
- Intent: Intentional
- Timing: Post-deployment
The incident was caused by malicious human actors who intentionally used deployed AI deepfake generation tools to create a fraudulent video for financial gain.
EU AI Act risk tier
- Risk tier: 3 Limited Risk
Limited Risk: The incident involves the creation of a deepfake video, which falls under Risk Level 3 (Limited Risk) as it is AI-generated content that requires transparency obligations to inform users of its synthetic nature.
AI system and alleged parties
- AI system: None named
- AI purpose: Deepfake Video Generation; Voice Generation
- Behaviour type: Tool
- Alleged developer: Unknown voice cloning technology developers, Unknown deepfake technology developers
- Alleged deployer: Unknown scammers, Sepenuhnya alami Facebook page
- Alleged harmed parties: Stipan Jonjić, People seeking medical advice, General public of Croatia, Epistemic integrity
Harm severity
Severity ratings are based on this scale.
Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.
- Physical: direct Negligible, indirect Negligible
- Infrastructure: direct Negligible, indirect Negligible
- Property: direct Negligible, indirect Negligible
- Financial: direct Negligible, indirect Negligible
- Environmental: direct Negligible, indirect Negligible
- Malicious content: direct Minor, indirect Minor
- Differential treatment: direct Negligible, indirect Negligible
- Civil rights: direct Negligible, indirect Negligible
- Democracy: direct Negligible, indirect Negligible
- Privacy: direct Negligible, indirect Negligible
- Psychological: direct Negligible, indirect Negligible
- Epistemic: direct Minor, indirect Minor
- Child sexual exploitation and abuse: direct Negligible, indirect Negligible
Malicious content
Reported: The report explicitly describes malicious content created and spread directly by the incident.
Directly caused: An AI-generated deepfake video of Dr. Stipan Jonjic was created and uploaded to Facebook to promote a fraudulent medical product.
Indirectly caused: The video was shared thousands of times on Facebook, spreading the malicious scam to a wide audience.
Epistemic
Reported: The report explicitly describes epistemic harm through the spread of medical misinformation and fake endorsements.
Directly caused: The deepfake video fabricated statements by Dr. Jonjic, falsely claiming that parasites cause HPV and most deaths, and that a specific herbal remedy cures them.
Indirectly caused: The video was shared thousands of times, potentially misleading viewers about medical facts and the doctor's actual endorsements.
People affected
- Occurrences reported: 1
- People reportedly harmed: 2
- People reportedly exposed: 1000
Potential causes
Technology
- AI Voice Cloning: AI tools replicated the doctor's voice with high accuracy.
- Deepfake Video Generation: AI synthesized lip movements to match fabricated speech.
- Deep Learning Tools: Advanced algorithms enable easy replication of human targets.
Data Inputs
- Accessible Training Data: Scammers used public YouTube videos to train the AI model.
Human Factors
- Trust in Authority: Victims believed the advice because it came from a known doctor.
- Detection Difficulty: Deepfakes are increasingly difficult for normal users to spot.
Process and Methods
- Weak Platform Moderation: The social network failed to flag or block the deepfake video.
- Unverified Page Ad Posting: Suspicious new pages can easily upload and distribute scam media.
Information quality
- Classification confidence: High
- Reason for confidence: The report clearly identifies the nature of the incident as an AI-generated deepfake scam, confirmed by the impersonated individual, and details the resulting financial harm to a consumer.
- Ambiguities identified: The specific AI tool used to generate the deepfake is not named, and the exact amount of money lost by the victim is not quantified.
- Alternative interpretations: None. The evidence clearly points to an intentional deepfake scam.
A deepfake video of a Croatian immunologist was used on Facebook to promote a fraudulent medical product, resulting in minor financial loss for a consumer. The incident represents a common cyber-enabled commercial fraud scam using AI tools, posing negligible threats to national security but highlighting the ongoing challenge of synthetic media in public information spaces.
- Overall national security impact: Minor
- Scope: Single nation
- Primary target: Croatia
- Alleged perpetrator: Unknown
Threat characteristics
- Imminence: Long-term. The incident represents an ongoing strategic concern regarding the proliferation of deepfakes rather than an active crisis.
- Autonomy: Human-controlled. AI tools were used strictly as instruments by human actors to generate synthetic media, with no autonomous decision-making.
- Novelty: Established threat. AI-generated deepfake scams targeting public figures for financial fraud are an established and frequently observed threat.
Impact by dimension
- Physical security: Negligible. No physical systems, infrastructure, or human safety capabilities were targeted or affected by this medical scam.
- Information security: Minor. While the incident involved AI-generated deepfake media, it was a localized financial scam rather than a coordinated state-sponsored information warfare campaign.
- Sovereignty: Negligible. The incident did not target or impact state authority, electoral processes, or core government decision-making.
- Economic security: Negligible. Financial loss was limited to individual scam victims purchasing a fake product, with no systemic threat to economic or technological security.
- Societal stability: Minor. The deepfake spread medical misinformation and identity theft of a public medical figure, causing minor localized concern but no large-scale instability.