Alleged AI-Generated Deepfake of Western Australia Premier Roger Cook Used in YouTube Investment Scam

Western Australia's Consumer Protection commissioner warned of a scam using a purported AI-generated deepfake of Premier Roger Cook to promote a fraudulent investment scheme in a YouTube pop-up ad. The manipulated video reportedly uses less than 10 seconds of real footage and voice and falsely depicts Cook endorsing "low investment, high return" opportunities. Cook called the scam "very scary," while regulators urged platforms to act on AI-driven frauds.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves malicious actors using AI-generated deepfakes of a public figure to execute a fraudulent investment scam targeting individuals for financial gain.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The harm is caused by human scammers who intentionally created and deployed a deepfake video post-deployment to defraud the public.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes AI-generated content such as deepfakes, which fall under Risk Level 3 due to transparency obligations requiring users to be informed they are interacting with or viewing AI-generated content.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: Unknown voice cloning technology developers, Unknown deepfake technology developers
• Alleged deployer: Unknown scammers impersonating Roger Cook, Unknown scammers
• Alleged harmed parties: Truth, Roger Cook, General public of Western Australia, General public of Australia, General public, Epistemic integrity, Australian investors

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: Yes, the report describes a malicious deepfake video and audio ad created to scam people.

Directly caused: An AI-generated deepfake video of the Premier was created and distributed as a pop-up ad on YouTube.

Epistemic

Reported: Yes, the report describes the creation of a highly convincing fake video and audio of the Premier making statements he never made.

Directly caused: The deepfake video fabricated statements of the Premier endorsing a financial scheme, misleading viewers who saw it.

Inferred additional harm: The proliferation of such indistinguishable deepfakes contributes to a broader erosion of trust in video evidence and digital information ecosystems.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 2

Potential causes

Management

• Profitization of Scam Advertisements: Platforms earn significant revenue from hosting unchecked fraudulent ads.
• Insufficient Safety Resource Allocation: Platforms do not deploy enough tools and data to block AI scams proactively.

Technology

• Sophisticated Deepfake Generation: AI requires only 10 seconds of voice/video to create highly realistic clones.
• Rapid Playback and Pop-up Delivery: Pop-up ads play quickly, preventing users from slowing down to spot flaws.
• High-Quality Audio-Visual Synthesis: Lip movements and voices are highly synchronized, making detection difficult.

Data Inputs

• Publicly Available Training Data: Abundant public video and audio of figures are harvested to train deepfakes.
• Minimal Source Data Requirement: Only 10 seconds of voice and video are needed to create convincing fakes.

Human Factors

• Inability to Detect Visual Flaws: Sophisticated deepfakes are nearly impossible for the human eye to discern.
• Susceptibility to Urgency Tactics: Scam ads create a false sense of urgency, bypassing critical thinking.
• Trust in Public Figure Likeness: Users trust endorsements from recognized leaders and authority figures.

Process and Methods

• Reactive Whack-a-Mole Takedowns: Scams are removed only after reports, allowing them to pop up repeatedly.
• Inadequate Pre-Ad Verification: Platforms fail to screen ads for deepfakes before they are published.
• Difficult Ad Interrogation: Ad formats prevent users from pausing or rewinding to analyze content.

Regulatory Environment

• Inadequate Platform Liability: Lack of strict regulation allows platforms to profit from fraudulent ads.
• Slow Regulatory Enforcement: Watchdogs rely on manual reports instead of proactive systemic bans.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly details the nature of the incident, which is an investment scam utilizing a deepfake of a public figure. The roles of the AI (generating the deepfake) and the humans (scammers deploying it) are clear. There is minimal ambiguity regarding the classification of this event as a scam (4.3).
• Ambiguities identified: The specific AI tool used to generate the deepfake is not identified, and the exact number of victims or financial losses is not quantified.
• Alternative interpretations: None. The event is unambiguously a deepfake-enabled financial scam.

An AI-generated deepfake video and voice clone of Western Australia Premier Roger Cook was used in a fraudulent YouTube investment scam. While distressing to the Premier and representing an evolved threat in digital deception, the incident remains a localized financial scam with minor direct national security implications.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: Australia
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic challenge of digital deception rather than an active crisis requiring immediate national security intervention.
• Autonomy: Human-controlled. The deepfake generator was utilized as a tool by human actors who controlled the inputs and deployment of the video.
• Novelty: Evolved capability. Deepfake scams are an established threat, but targeting a sitting Premier with highly realistic video and voice cloning represents an evolved capability.

Impact by dimension

• Physical security: Negligible. No physical security, critical infrastructure, or kinetic targeting systems were affected by this incident.
• Information security: Minor. The incident involved synthetic media of a political figure, but was deployed for financial fraud rather than coordinated state-sponsored information warfare or intelligence compromise.
• Sovereignty: Minor. The deepfake targeted a state Premier's likeness but did not disrupt core government operations, electoral processes, or sovereign decision-making.
• Economic security: Minor. The scam targeted individual consumers for financial fraud but did not pose a threat to strategic industries, critical supply chains, or national financial stability.
• Societal stability: Minor. The unauthorized use of a political leader's likeness could erode public trust in official communications, but did not cause large-scale civil unrest or systemic rights violations.