Sexually suggestive videos featuring faces of female celebrities such as Emma Watson and Scarlett Johansson were rolled out as ads on social media for an app allowing users to create deepfakes.
.avif){kind=small}
The Facemega app, developed by Ufoto Limited, was used to run hundreds of sexually suggestive advertisements on Meta platforms and other apps. These ads featured deepfake face-swaps of celebrities like Emma Watson and Scarlett Johansson to promote the app's ability to generate nonconsensual sexual content. Following public outcry and media reports, the app was removed from the Apple App Store and Meta's advertising services, though it remained available on Google Play for a period.
Sexually suggestive videos featuring faces of female celebrities such as Emma Watson and Scarlett Johansson were rolled out as ads on social media for an app allowing users to create deepfakes.
The incident involved running hundreds of public advertisements featuring sexually suggestive deepfakes of celebrities, exposing millions of social media users to toxic and inappropriate content.
The incident was caused by human developers intentionally creating and running suggestive deepfake ad campaigns to promote their deployed AI application.
Limited Risk: The report describes an AI system used to generate deepfakes, which falls under the Limited Risk category of the EU AI Act, requiring specific transparency obligations to inform users.
Severity ratings are based on this scale.
Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.
Reported: The reports explicitly describe the creation and distribution of toxic and sexually suggestive deepfake content.
Directly caused: Over 230 suggestive deepfake ads featuring celebrity face-swaps were distributed across Meta platforms, with one viral tweet of the ad reaching over 10 million views.
Indirectly caused: The app facilitates and encourages users to generate their own nonconsensual sexually explicit or suggestive deepfake videos.
Inferred additional harm: Thousands of additional toxic, nonconsensual sexual videos generated by the app's user base.
Reported: The reports explicitly describe how deepfake technology is disproportionately used as a weapon to target and harass women and girls.
Directly caused: The promotional campaign specifically targeted and sexualized prominent female celebrities.
Indirectly caused: The proliferation of easy-to-use deepfake apps disproportionately subjects women and girls to nonconsensual sexualization and online harassment.
Inferred additional harm: Systemic increase in gender-based harassment and unequal safety outcomes for women online.
Reported: The reports explicitly describe violations of personal autonomy and consent.
Directly caused: The nonconsensual sexualization of Emma Watson and Scarlett Johansson violated their basic rights to bodily autonomy and personal dignity.
Indirectly caused: The app enables mass violations of individual consent by allowing users to generate sexual content of anyone without permission.
Inferred additional harm: Widespread violations of civil rights and personal dignity for thousands of individuals targeted by users of the app.
Reported: The reports explicitly describe privacy concerns arising from the nonconsensual use of personal likenesses.
Directly caused: The unauthorized use of celebrity faces in suggestive promotional videos violated their personal privacy.
Indirectly caused: The app allows users to upload photos of any private individual without consent, facilitating severe privacy violations.
Inferred additional harm: Widespread privacy violations of private citizens whose personal photos are uploaded and manipulated without their knowledge.
Reported: The reports explicitly describe psychological harm, including humiliation, distress, and violation of bodily autonomy.
Directly caused: The nonconsensual sexualization of Emma Watson and Scarlett Johansson in public ads caused reputational damage and distress.
Indirectly caused: The widespread availability of the app causes anxiety, disgust, and fear of harassment among women and girls who worry about being targeted.
Inferred additional harm: Widespread psychological trauma and distress for numerous unnamed victims whose likenesses have been exploited using this app.
Reported: The reports explicitly describe concerns regarding the erosion of shared reality and the difficulty of distinguishing fact from fiction.
Directly caused: The ads presented highly realistic but entirely fabricated videos of celebrities in suggestive scenarios.
Indirectly caused: The proliferation of highly convincing deepfakes undermines public trust in video evidence and visual media.
Inferred additional harm: Broader societal degradation of trust in digital media as deepfake technology becomes indistinguishable from reality.
A commercial deepfake application, Facemega, ran aggressive ad campaigns featuring nonconsensual celebrity face-swaps on major social platforms. While highlighting the rapid commercialization and accessibility of realistic synthetic media tools, the incident represents a minor national security threat, primarily impacting digital privacy, platform safety, and consumer protection rather than critical state infrastructure or functions.