Deepfake Videos of Barbara O’Neill Allegedly Used in Health Scam Targeting Social Media Users

Deepfake videos of Barbara O’Neill, an Australian alternative health advocate banned from giving medical advice, have allegedly been used in fraudulent social media ads promoting false health cures. They include treatments for prostate issues and erectile dysfunction. These manipulated videos have been widely circulated across African countries. The scam directs users to deceptive websites, potentially for phishing or financial fraud. O’Neill has denied involvement.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves scammers using AI-generated deepfakes of a public figure to deceive social media users for financial gain through phishing and selling unverified supplements.

Additional risk subdomains

• 3.1 False or misleading information: The deepfake videos spread false health claims and medical misinformation regarding prostate cures.
• 2.1 Compromise of privacy by obtaining, leaking or correctly inferring sensitive information: The scam utilizes phishing websites designed to harvest personal information from unsuspecting users.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The risk is caused by human scammers intentionally deploying AI deepfake tools post-deployment to defraud social media users.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the creation and dissemination of 'AI-generated impersonations' and 'deepfakes', which fall under the transparency obligations of Risk Level 3.

AI system and alleged parties

• AI system: None named
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: Unknown voice cloning technology developers, Unknown deepfake technology developers
• Alleged deployer: Unknown scammers, Scammers impersonating Barbara O'Neill
• Alleged harmed parties: social media users, Individuals with health problems, Citizens of various African nations, Barbara O'Neill

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Minor
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes malicious content created and spread directly by the incident.

Directly caused: Scammers created and spread AI-generated deepfake videos of [PERSON_002] promoting fake cures, which accumulated over a million views on Facebook.

Privacy

Reported: The report explicitly describes potential privacy violations through phishing scams linked to the deepfake videos.

Indirectly caused: The advertisements directed users to fake health websites designed to harvest personal information in phishing scams.

Inferred additional harm: It is highly likely that numerous users entered their personal information on these fake sites, resulting in unauthorized access to and compromise of their sensitive data.

Epistemic

Reported: The report explicitly describes epistemic harm caused directly by the incident through the spread of medical misinformation.

Directly caused: The deepfake videos generated and spread false claims that a specific capsule could cure prostatitis and restore erections.

Inferred additional harm: The widespread circulation of these deepfakes contributes to the erosion of public trust in legitimate medical institutions and the pollution of the online information ecosystem.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 1000000

Potential causes

Management

• Flawed Ad Approval Process: Facebook allowed coordinated ad campaigns using deepfakes to run unchecked.
• Inadequate Content Takedown Speed: Platform management failed to quickly remove reported deepfake scam ads.

Technology

• Accessible AI Deepfake Tools: AI tools enable rapid creation of low-quality but convincing voice clones.
• Audio-Video Desynchronization: AI-generated audio fails to sync perfectly with original video movements.
• Robotic Audio Synthesis: Synthesized voices sound unnatural and robotic in certain segments.

Data Inputs

• Abundant Public Lecture Footage: Large volumes of online lecture videos provide ample source data for training.
• Unprotected Personal Identity Data: Publicly available audio clips of the speaker are harvested to train AI.

Human Factors

• User Susceptibility to Scams: Audiences trust familiar figures and fall for sensational health claims.
• Targeting Vulnerable Demographics: Scammers exploit people seeking quick cures for sensitive health issues.
• Overreliance on Social Media Content: Users assume hosted advertisements on major platforms are pre-screened.

Process and Methods

• Ineffective Ad Moderation: Social media platforms fail to detect and block deepfake scam ads.
• Lack of Proactive Content Filtering: Platforms rely on reactive reporting rather than proactive AI detection.
• Facile Account Creation Processes: Scammers easily set up multiple small accounts to run coordinate ads.

Regulatory Environment

• Ineffective Online Ban Enforcement: Authorities banned the speaker but cannot stop her content spreading online.
• Lack of Platform Accountability: No regulatory framework holds social media liable for hosting deepfake scams.

Information quality

• Classification confidence: High
• Reason for confidence: The fact-checking report from Africa Check provides clear, reliable details about the nature of the deepfake scam, the platforms used, the geographic reach, and the specific deceptive tactics employed. While the exact AI models are not named, this does not impede the classification of the incident as a deepfake-enabled scam.
• Ambiguities identified: The specific AI software or platforms used to generate the deepfakes are not identified in the report.
• Alternative interpretations: None.

Fraudulent advertising campaigns utilized AI-generated deepfakes of an Australian health advocate to promote fake medical cures and direct users to phishing sites across several African nations. The incident represents a minor national security concern, categorized primarily as consumer fraud and localized cybercrime rather than state-sponsored information warfare.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: South Africa, Nigeria, Kenya, Zimbabwe
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing cybercrime and misinformation concern rather than an imminent national security crisis.
• Autonomy: Human-controlled. AI tools were utilized by human threat actors to generate content, with no autonomous decision-making or execution by the system.
• Novelty: Established threat. Deepfake-enabled phishing and product scams are well-established methodologies in cybercrime.

Impact by dimension

• Physical security: Negligible. No impact on critical infrastructure, physical systems, or kinetic military capabilities.
• Information security: Minor. Involves AI-generated deepfakes used for fraudulent advertising, but lacks the strategic coordination of state-sponsored information warfare.
• Sovereignty: Negligible. No impact on government decision-making, electoral systems, or core state sovereignty functions.
• Economic security: Negligible. Represents consumer fraud and localized phishing rather than a threat to national financial systems or strategic technologies.
• Societal stability: Minor. The campaign exploits an individual likeness and targets citizens across multiple nations with medical misinformation, but does not threaten overall societal stability.