Purported Pornographic Deepfakes and Fake Accounts Reportedly Impersonated German TV Presenter and Actor Collien Fernandes

Collien Fernandes, a German television presenter and actor, accused her former husband, Christian Ulmen, of carrying out a years-long online impersonation campaign that allegedly used fake accounts and AI-generated pornographic deepfakes resembling her. The allegations include sexually explicit synthetic images and videos presented as if they were hers, while Ulmen's lawyer reportedly denied the claims and said he would take legal action over the reporting.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The perpetrator used AI deepfakes and voice cloning for targeted manipulation and humiliation of a specific victim, creating non-consensual sexual imagery to exploit her likeness.

Additional risk subdomains

• 1.2 Exposure to toxic content: The incident involved the creation and distribution of highly toxic, non-consensual pornographic deepfakes to hundreds of individuals online.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The harm was caused by the deliberate and intentional actions of a human perpetrator who used deployed AI technologies to harass and impersonate the victim.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. The incident involves the creation of 'AI-generated content such as deepfakes' and voice cloning, which are subject to transparency obligations under the EU AI Act.

AI system and alleged parties

• AI system: unspecified AI generator
• AI purpose: Deepfake Video Generation; Voice Generation
• Behaviour type: Tool
• Alleged developer: Unknown deepfake technology developers
• Alleged deployer: Christian Ulmen
• Alleged harmed parties: Epistemic integrity, Collien Fernandes

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Substantial, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Minor, indirect Minor
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Malicious content

Reported: The report explicitly describes the creation and distribution of toxic, non-consensual pornographic deepfakes.

Directly caused: The perpetrator generated explicit pornographic videos and photos of the victim and distributed them to hundreds of men, with one video receiving over 270,000 views.

Inferred additional harm: It is highly likely that the explicit deepfakes continue to circulate on various adult websites, exposing more internet users to non-consensual toxic content.

Privacy

Reported: The report explicitly describes privacy violations, including identity theft and disclosure of secrets.

Directly caused: The perpetrator stole the victim's identity by creating fake profiles in her name and using her likeness and cloned voice without her consent.

Psychological

Reported: The report explicitly describes severe psychological and emotional distress suffered by the victim.

Directly caused: The victim described the confession of her ex-husband as 'like receiving a news of death' and stated that her 'world collapsed' and her 'body was stolen from me for years'.

Inferred additional harm: The victim likely suffered long-term psychological trauma, anxiety, and distress requiring professional mental health support. The 30 men who were deceived into intense relationships also likely suffered emotional distress.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 301

Potential causes

Management

• Lax Platform Moderation: Online platforms failed to block accounts or detect synthetic explicit content.
• Unrestricted AI Image Generation: AI platforms like Grok lacked safeguards against generating sexual images.

Technology

• AI Deepfake Generation Tools: Allowed creation of realistic pornographic videos and photos of the victim.
• AI Voice Cloning Technology: Enabled emulation of victim's voice for phone sex with multiple men.

Data Inputs

• Unauthorized Use of Personal Images: Victim's photos were used to train or guide AI models.
• Voice Sample Exploitation: Ex-husband used existing voice recordings to generate synthetic speech.

Human Factors

• Perpetrator's Desire for Control: Ex-husband sought power, humiliation, and pleasure through digital abuse.
• Victim's Initial Misattribution: Victim assumed anonymous trolls were behind the attacks for many years.

Process and Methods

• Weak Identity Verification: Online platforms allowed creation of fake profiles under the victim's name.
• Ineffective Content Takedown: Lawyers failed to successfully remove the deepfake videos from the web.

Regulatory Environment

• German Legal Loopholes: German law did not criminalize the production of pornographic deepfakes.
• Lack of Digital Violence Laws: Germany lacked specific legal frameworks to protect women from digital abuse.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent and detailed accounts of the incident from multiple reputable news sources. The roles of the perpetrator, the victim, and the AI technologies used are clearly defined, and the legal proceedings are well-documented.
• Ambiguities identified: The specific AI models or platforms used by the perpetrator to generate the deepfakes and clone the voice are not identified.
• Alternative interpretations: None. The facts of the impersonation and deepfake creation are admitted by the perpetrator according to the reports.

A high-profile case of digital violence where a German actor used AI deepfakes and voice cloning to harass and impersonate his ex-wife. While causing severe personal and psychological harm and prompting national legislative debate in Germany, the incident remains a domestic dispute with minor national security implications.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: Germany
• Other affected: Spain
• Alleged perpetrator: Christian Ulmen

Threat characteristics

• Imminence: Long-term. The perpetrator has confessed and legal proceedings are underway, representing a resolved domestic incident rather than an active national crisis.
• Autonomy: Human-controlled. The AI systems for voice cloning and deepfakes were used strictly as tools by the human perpetrator to generate specific content.
• Novelty: Evolved capability. Combining highly realistic voice cloning and deepfakes for sustained personal impersonation represents an evolved capability, though deepfake-based harassment is an established threat.

Impact by dimension

• Physical security: Negligible. No physical security threats, kinetic attacks, or critical infrastructure disruptions occurred during this incident.
• Information security: Negligible. The deepfakes and voice cloning were used for personal harassment and domestic abuse, not state-sponsored information warfare or intelligence compromise.
• Sovereignty: Negligible. No impact on state authority, electoral systems, or core government operations. The subsequent legislative debate in Germany is a standard democratic process.
• Economic security: Negligible. No impact on financial systems, strategic technologies, or critical supply chains.
• Societal stability: Minor. Represents a severe case of digital violence and privacy violation against an individual, prompting national legislative debate, but remains limited to a personal dispute without widespread societal disruption.