Old Mutual and former finance minister Trevor Manuel have issued warnings regarding deepfake videos circulating on social media. These videos use Manuel's likeness and a cloned voice to promote fraudulent investment schemes. The incident highlights the ongoing use of synthetic media by bad actors to deceive the public into participating in financial scams.
Old Mutual of South Africa reportedly warned of purported deepfake videos impersonating its chairman, former finance minister Trevor Manuel, promoting fake investment schemes on social media. The purported AI-generated videos used his likeness and cloned voice to solicit funds, prompting a public statement from Manuel denying involvement.
Risk classification
- Primary risk domain: 4 Malicious actors
- Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation
The incident involves the targeted exploitation of specific victims for financial gain using synthetic media to impersonate a trusted public figure.
Additional risk subdomains
- 3.1 False or misleading information: The deepfake videos generate false information by depicting Trevor Manuel giving unauthorized financial advice.
Causal factors
- Entity: Human
- Intent: Intentional
- Timing: Post-deployment
The risk and subsequent harm were caused by malicious human actors intentionally deploying deepfake technology to defraud social media users.
EU AI Act risk tier
- Risk tier: 3 Limited Risk
Limited Risk: The report describes the use of AI-generated content, specifically deepfakes, which falls under Risk Level 3 due to transparency obligations to inform users they are interacting with AI-generated media.
AI system and alleged parties
- AI system: None named
- AI purpose: Deepfake Video Generation; Voice Generation
- Behaviour type: Tool
- Alleged developer: Unknown voice cloning technology developers, Unknown deepfake technology developers
- Alleged deployer: Unknown scammers impersonating Trevor Manuel, Unknown scammers
- Alleged harmed parties: Truth, Trevor Manuel, Old Mutual, Investors, General public of South Africa, Epistemic integrity
Harm severity
Severity ratings are based on this scale.
Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.
- Physical: direct Negligible, indirect Negligible
- Infrastructure: direct Negligible, indirect Negligible
- Property: direct Negligible, indirect Negligible
- Financial: direct Negligible, indirect Negligible
- Environmental: direct Negligible, indirect Negligible
- Malicious content: direct Minor, indirect Negligible
- Differential treatment: direct Negligible, indirect Negligible
- Civil rights: direct Negligible, indirect Negligible
- Democracy: direct Negligible, indirect Negligible
- Privacy: direct Negligible, indirect Negligible
- Psychological: direct Negligible, indirect Negligible
- Epistemic: direct Minor, indirect Negligible
- Child sexual exploitation and abuse: direct Negligible, indirect Negligible
Malicious content
Reported: The report explicitly describes malicious content created and spread directly by the incident.
Directly caused: Deepfake videos and artificial voice clones of Trevor Manuel were created and circulated on social media platforms to promote fraudulent investments.
Inferred additional harm: It is likely that similar deepfake content targeting other high-profile individuals like Johann Rupert and Nicky Oppenheimer reached thousands of social media users.
Epistemic
Reported: The report explicitly describes epistemic harm through the fabrication of video and voice evidence.
Directly caused: The AI-generated videos fabricated statements of Trevor Manuel giving investment advice, which is illegal and false.
Inferred additional harm: The proliferation of such deepfakes erodes public trust in video evidence and the authentic communications of public figures.
People affected
- Occurrences reported: 1
- People reportedly harmed: 10
- People reportedly exposed: 1000
Potential causes
Management
- Inadequate Platform Oversight: Social media platforms failed to prevent scam ads and deepfakes.
Technology
- Deepfake Video Generation: AI used to create realistic fake videos of prominent figures.
- Artificial Voice Synthesis: AI voice cloning used to mimic the target's voice convincingly.
- Unverified AI Trading Software: Claims of automated AI trading software making high-profit decisions.
Data Inputs
- Unauthorized Image Use: Scammers used Trevor Manuel's likeness and images without consent.
- Fictitious Trading Patterns: Sharing fake trading patterns and signals to lure clients.
Human Factors
- Public Trust in Public Figures: Victims trust advice associated with well-known financial figures.
- Susceptibility to High Returns: Investors lured by promises of unrealistic or exaggerated returns.
- Lack of Verification Habits: Failure of individuals to verify FSP registration before investing.
Process and Methods
- Social Media Distribution: Scams distributed easily through social media and Telegram.
- Lack of Content Moderation: Social media platforms failed to detect and block deepfake scam videos.
Regulatory Environment
- Unlicensed Financial Advice: Unauthorised entities providing financial services without FSCA approval.
- Regulatory Enforcement Gaps: Difficulty in rapidly policing and shutting down online scams.
Information quality
- Classification confidence: High
- Reason for confidence: The report clearly describes the nature of the deepfake scam, the target (Trevor Manuel), and the medium (social media). While specific financial loss figures and victim counts are missing, the classification of the AI risk domain and causal factors is highly straightforward.
- Ambiguities identified: The specific AI tools used to generate the deepfakes are not identified.
- Alternative interpretations: None.
Malicious actors utilized deepfake video and voice-cloning technology to impersonate former South African Finance Minister Trevor Manuel in order to promote fraudulent investment schemes. While the incident highlights the growing threat of synthetic media to public trust and financial security, it represents a localized criminal scam rather than a significant national security threat.
- Overall national security impact: Minor
- Scope: Single nation
- Primary target: South Africa
- Alleged perpetrator: Unknown
Threat characteristics
- Imminence: Long-term. Represents an ongoing strategic challenge of synthetic media abuse rather than an immediate, high-priority national security crisis.
- Autonomy: Human-controlled. AI systems were used as basic tools by human operators to generate deceptive video and voice clones.
- Novelty: Established threat. Deepfake financial scams targeting high-profile figures are a well-documented and frequently observed threat vector.
Impact by dimension
- Physical security: Negligible. No threat to physical systems, critical infrastructure, or human safety was reported in connection with this incident.
- Information security: Minor. Involves synthetic media and voice cloning of a public figure, but used for localized financial fraud rather than state-sponsored disinformation or intelligence compromise.
- Sovereignty: Minor. Impersonation of a former finance minister is concerning, but did not disrupt core government operations, electoral systems, or state authority.
- Economic security: Minor. Causes financial losses to individuals through fraudulent investment schemes, but does not pose a systemic risk to the national financial system or strategic industries.
- Societal stability: Minor. Erodes public trust through synthetic media and causes financial harm to citizens, but does not threaten large-scale social cohesion or civil liberties.