Purported Synthesia Avatars Used in Alleged Pro-Junta Deepfake Video Supporting Ibrahim Traoré in Burkina Faso

In January 2023, purportedly AI-generated videos reportedly began circulating on WhatsApp and social media showing avatars of apparent American pan-Africanists expressing support for Burkina Faso's military junta. The videos were reportedly created using the Synthesia platform. Synthesia later confirmed the videos violated its terms of service and banned the user. The source of the videos remains unidentified.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.1 Disinformation, surveillance, and influence at scale

The incident involved a user leveraging Synthesia's AI video generation to create and spread pro-military junta propaganda, aiming to influence public opinion and political processes in Burkina Faso.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The risk was caused by a human user who intentionally generated and distributed pro-military propaganda videos using Synthesia's post-deployment platform.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. The incident involves 'AI-generated content such as deepfakes' created via Synthesia, which falls under the transparency obligations of Limited Risk systems.

AI system and alleged parties

• AI system: Synthesia
• AI purpose: Deepfake Video Generation; Social Media Content Generation
• Behaviour type: Tool
• Alleged developer: Synthesia
• Alleged deployer: Unknown actors, Pro-Ibrahim Traoré information ecosystem
• Alleged harmed parties: Truth, National security and intelligence stakeholders, General public of West Africa, General public of Burkina Faso, General public, Epistemic integrity

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Minor
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Minor, indirect Minor
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the creation and spread of pro-military junta propaganda videos using AI-generated avatars.

Directly caused: At least two AI-generated videos featuring Synthesia avatars reading a script supporting Burkina Faso's military junta were created and shared.

Indirectly caused: The videos were circulated on WhatsApp, posted on Facebook, and shared widely on Twitter.

Inferred additional harm: It is likely that additional similar propaganda videos were created or shared within closed networks like WhatsApp that were not publicly documented.

Democracy

Reported: The report explicitly describes an attempt to spread pro-military propaganda to support a military junta that seized power in a coup.

Directly caused: AI-generated avatars were used to call on the Burkinabe people to 'effectively support the authorities of the transition' following a military coup.

Indirectly caused: The circulation of synthetic political propaganda on social media platforms potentially undermines democratic discourse and legitimizes undemocratic power grabs.

Inferred additional harm: The ongoing use of synthetic media in conflict zones or politically unstable regions could further erode trust in democratic institutions and transition processes.

Epistemic

Reported: The report explicitly describes the spread of synthetic videos designed to look like real people expressing political support.

Directly caused: AI-generated avatars were used to fabricate statements of support from a self-proclaimed 'Pan-Africanist' named 'Emilienne'.

Indirectly caused: The videos misled at least one political scientist who shared the video on Facebook without realizing it was AI-generated.

Inferred additional harm: The proliferation of such deepfakes contributes to the erosion of a shared consensus reality, making it harder for citizens to distinguish between genuine and synthetic political expressions.

People affected

• Occurrences reported: 1
• People reportedly exposed: 100

Potential causes

Management

• Inadequate Risk Assessment: Management failed to prevent geopolitical exploitation of their avatar tool.

Technology

• AI Avatar Generation Abuse: Synthesia's platform was used to generate lifelike political propaganda.
• Inadequate Algorithmic Detection: Automated systems failed to flag the script due to lack of explicit terms.

Data Inputs

• Implicit Propaganda Scripts: Input text avoided explicit banned words to bypass keyword filters.

Human Factors

• Lack of Public AI Awareness: Users shared the video on social media without realizing it was AI-generated.
• Malicious User Intent: An anonymous actor intentionally used AI to spread pro-military propaganda.

Process and Methods

• Weak Contextual Moderation: Moderation lacked contextual understanding of regional political situations.
• Reactive Moderation Process: Platform only banned the user after the video was flagged externally.

Regulatory Environment

• Lack of Deepfake Regulation: Absence of strict legal frameworks governing AI-generated political content.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly identifies the AI platform used (Synthesia), the nature of the content generated (pro-military junta propaganda), and the platforms where it circulated. Synthesia's CEO confirmed the misuse and the subsequent ban of the user, leaving little ambiguity about the technical tool or the violation of terms of service.
• Ambiguities identified: The identity of the creator of the videos remains unknown, and there is unverified speculation regarding the involvement of the Wagner Group.

AI-generated avatars were used to create clumsy political propaganda videos supporting Burkina Faso's military junta. While representing an evolution in disinformation tactics, the low quality of the videos and the swift ban of the creator limited the national security impact to a minor level.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: Burkina Faso
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The specific incident has been resolved with the user banned, representing an ongoing strategic concern regarding synthetic propaganda rather than an active crisis.
• Autonomy: Human-controlled. The AI system operated strictly as a tool, generating video content based on direct text inputs provided by a human user.
• Novelty: Evolved capability. Represents an advancement of existing online disinformation tactics by integrating easily accessible AI video generation tools to create political propaganda.

Impact by dimension

• Physical security: Negligible. No kinetic threats, physical infrastructure manipulation, or physical safety risks were reported in connection with this incident.
• Information security: Minor. AI-generated avatars were used for political propaganda supporting a military junta. However, the videos were clumsy, unlikely to deceive many, and quickly removed.
• Sovereignty: Minor. The videos supported a military junta that had already seized power, but there was no direct compromise of core government functions or electoral systems.
• Economic security: Negligible. No financial systems, strategic technologies, or critical supply chains were targeted or affected by this incident.
• Societal stability: Minor. The propaganda aimed to influence public support for an authoritarian junta, potentially affecting democratic norms, but had limited actual societal impact.