Purportedly AI-Generated Videos Impersonate Brunei Police in 'Real Money Magic' Scam on Social Media

The Royal Brunei Police Force issued a public warning about purportedly AI-generated deepfake videos impersonating police officers on TikTok, Facebook, and Instagram. The videos allegedly promote a fraudulent investment scheme known as "Real Money Magic," reportedly claiming guaranteed financial returns. Authorities report that scammers use synthetic video and audio to gain victims' trust and induce money transfers. The scam has not been linked to any official institutions.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves scammers using AI-generated deepfakes to impersonate police officers and run a fraudulent investment scam targeting individuals for financial gain.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The risk was caused by human scammers who intentionally used deployed AI tools to generate deepfakes for fraudulent purposes.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. The report describes AI-generated content such as deepfakes, which are subject to specific transparency obligations under the Act.

AI system and alleged parties

• AI system: None named
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: Unknown voice cloning technology developers, Unknown deepfake technology developers
• Alleged deployer: Unknown scammers targeting the Royal Brunei Police Force, Unknown scammers
• Alleged harmed parties: Royal Brunei Police Force, General public of Brunei

Harm severity

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes deepfake videos and audio circulating on social media platforms.

Directly caused: Scammers created and spread deepfake videos and audio impersonating police officers on TikTok, Facebook, and Instagram.

Indirectly caused: N/A

Inferred additional harm: Additional deepfake videos may have been generated and shared across other messaging platforms like WhatsApp, exposing more users to the malicious content.

Epistemic

Reported: The report describes AI-generated deepfakes of police officers used to mislead the public.

Directly caused: The creation of highly convincing fake videos and audio of real police personnel undermined the public's ability to trust official communications.

Indirectly caused: N/A

Inferred additional harm: The proliferation of such deepfakes could lead to a broader erosion of trust in law enforcement and digital media authenticity among the Brunei public.

People affected

• Occurrences reported: 1
• People reportedly exposed: 1000

Potential causes

Technology

• AI Deepfake Generation: AI tools used to create realistic manipulated video and audio of police.
• Social Media Distribution: Platforms like TikTok and Facebook allow rapid circulation of deepfakes.

Data Inputs

• Misuse of Officer Likeness: Scammers use real police officers' images and voices to train AI models.

Human Factors

• Exploitation of Public Trust: Scammers exploit the trust and authority associated with police officers.
• Vulnerability to Financial Scams: Victims are lured by promises of quick profits and guaranteed returns.

Process and Methods

• Inadequate Content Moderation: Social media platforms fail to detect and remove fraudulent deepfake videos.

Regulatory Environment

• Lack of AI Content Controls: Absence of strong regulations to prevent the malicious deployment of deepfakes.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly outlines the nature of the scam, the platforms used, the impersonated entity (police officers), and the specific fraudulent scheme ('Real Money Magic'). There is little ambiguity about the malicious intent or the technology used (AI deepfakes).

The Royal Brunei Police Force issued a warning regarding AI-generated deepfakes impersonating police officers to promote a fraudulent investment scheme. While the incident represents an exploitation of state authority and trust, its national security impact is minor, primarily constituting localized financial cybercrime rather than a coordinated threat to state sovereignty or critical infrastructure.

• Overall national security impact: Minor
• Response level: Moderate
• Scope: Single nation
• Primary target: Brunei
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Near-term. The scam is actively circulating on social media, requiring near-term monitoring and public warnings to mitigate ongoing financial victimization.
• Autonomy: Human-controlled. The AI systems were used as tools by human scammers to generate manipulated media, with no autonomous decision-making or execution.
• Novelty: Established threat. AI-generated deepfakes for financial fraud and impersonation are well-established global threats, representing a standard application of existing technology.

Impact by dimension

• Physical security: Negligible. No physical threat, kinetic attacks, or critical infrastructure compromise reported in relation to these financial scam deepfakes.
• Information security: Minor. Use of AI-generated deepfakes to impersonate police officers represents a minor information manipulation threat, though aimed at financial fraud rather than coordinated state-sponsored warfare.
• Sovereignty: Minor. Impersonation of law enforcement officers exploits state authority but does not compromise core government decision-making, electoral systems, or sovereign control.
• Economic security: Negligible. The incident involves localized financial fraud targeting individuals, with no systemic threat to Brunei's financial systems, strategic industries, or critical supply chains.
• Societal stability: Minor. The deepfakes erode public trust in official law enforcement communications, presenting a minor risk to societal trust, though limited to a localized financial scam.