Deepfake of FTX's Former CEO Posted on Twitter Aiming to Scam FTX Collapse Victims

A visual and audio deepfake of former FTX CEO Sam Bankman-Fried was posted on Twitter to scam victims of the exchange's collapse by urging people to transfer funds into an anonymous cryptocurrency wallet.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves the use of a deepfake to impersonate a public figure for the purpose of defrauding victims of the FTX collapse.

Additional risk subdomains

• 3.1 False or misleading information: The deepfake generated entirely fabricated statements claiming that FTX was hosting a cryptocurrency giveaway to compensate users.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was caused by a human actor who intentionally created and deployed a deepfake of Sam Bankman-Fried to execute a financial scam.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the creation of a deepfake, which falls under Risk Level 3 as it requires transparency obligations to inform users they are interacting with AI-generated content.

AI system and alleged parties

• AI system: None named
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: unknown
• Alleged deployer: unknown
• Alleged harmed parties: victims of FTX's collapse, Twitter Users

Harm severity

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 1000

Potential causes

Management

• Weak Platform Content Moderation: Twitter management failed to proactively block deepfake scam campaigns.
• Inadequate Risk Assessment: Failure to assess risks of paid verification policy exploitation.

Technology

• Convincing Deepfake Generation: ML and AI techniques generated realistic visual and audio content of SBF.
• Malicious Scam Website: A live phishing website was used to collect cryptocurrency from victims.

Data Inputs

• SBF Audio and Video Data: Abundant public media of SBF allowed training of realistic deepfake models.

Human Factors

• Victim Desperation for Recovery: FTX collapse victims were desperate to recover their lost assets.
• Trust in Twitter Verification: Users trusted the blue checkmark on the malicious account.

Process and Methods

• Flawed Twitter Verification Policy: Twitter's paid verification allowed scammers to obtain verified status easily.
• Delayed Account Suspension: The scam Twitter account remained active long enough to post the deepfake.

Regulatory Environment

• Lack of Deepfake Regulation: Absence of strict legal frameworks governing the creation of deepfakes.
• Unregulated Crypto Platforms: Lack of oversight on crypto transactions enabled rapid transfer of funds.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly details the nature of the scam, the specific deepfake used, the platform (Twitter), and the financial impact (0.89 ETH). The role of the AI as a generative tool is clear, and the boundary rules easily resolve the classification.
• Ambiguities identified: The specific AI tool used to generate the deepfake is not identified.

A deepfake video of former FTX CEO Sam Bankman-Fried was used in a minor cryptocurrency scam on Twitter. The incident resulted in negligible financial loss and poses no threat to national security, representing an established cybercrime method rather than a state-level threat.

• Overall national security impact: Negligible
• Response level: Minor
• Scope: Unknown
• Primary target: No clear primary
• Other affected: Unknown
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The specific incident is resolved and the account suspended, representing a long-term strategic concern regarding deepfake technology rather than an active crisis.
• Autonomy: Human-controlled. The AI system was used as a generative tool directly controlled by a human actor to create the synthetic media.
• Novelty: Established threat. The use of deepfakes for financial scams and cryptocurrency giveaways is an established threat method with multiple prior occurrences.

Impact by dimension

• Physical security: Negligible. The incident is a digital financial scam with no physical impact, kinetic threat, or damage to critical infrastructure.
• Information security: Negligible. The deepfake was used for a localized financial scam targeting retail investors, with no connection to state-sponsored information warfare or intelligence compromise.
• Sovereignty: Negligible. The scam targeted private individuals and did not impact government operations, electoral systems, or state sovereignty.
• Economic security: Negligible. The total reported financial loss was approximately $1,006, representing a negligible impact on national economic or technological security.
• Societal stability: Negligible. The incident was a targeted financial scam affecting a very small number of individuals, posing no threat to broader societal stability or human rights.