Novel Deepfake Song Pulled from Music Streaming Services After Allegedly Violating Artist's Rights

The deepfake performance of "Heart On My Sleeve" created to mimic the voice and musical styles of Drake and The Weeknd is no longer available on several streaming services after their record label served copyright takedown notices to the platforms.

Risk classification

• Primary risk domain: 6 Socioeconomic & Environmental
• Primary risk subdomain: 6.3 Economic and cultural devaluation of human effort

The incident involves the reproduction and imitation of the artists' unique voices and creative styles, which threatens to destabilize the music industry and deny artists their due compensation.

Additional risk subdomains

• 4.3 Fraud, scams, and targeted manipulation: The creator impersonated the artists' voices to distribute a fake track on commercial streaming platforms for financial gain.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The incident was caused by the intentional decision of a human creator, [HANDLE_001], to use deployed generative AI voice-cloning software to mimic the artists and distribute the track.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The incident involves the creation of deepfakes and AI-generated audio content mimicking real individuals, which triggers specific transparency and notification obligations under the Act.

AI system and alleged parties

• AI system: unspecified
• AI purpose: Voice Generation; Text Style Replication
• Behaviour type: Tool
• Alleged developer: unknown
• Alleged deployer: @ghostwriter
• Alleged harmed parties: Universal Music Group, The Weeknd, Drake

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 1
• People reportedly harmed: 2
• People reportedly exposed: 2

Potential causes

Management

• Platform Conflict of Interest: Google faces tension between protecting YouTube partners and training Bard AI.
• Prioritizing Growth Over Safety: Streaming platforms hosted viral AI content before establishing safety rules.

Technology

• Generative AI Voice Cloning: Software allowed high-quality replication of artists' voices without consent.
• ContentID Limitations: Automated systems cannot detect AI clones because labels do not own the track.

Data Inputs

• Unauthorized Training Data: AI model was trained on proprietary artist catalogs without licenses.

Human Factors

• Creator Hype Generation: Ghostwriter977 uploaded and re-uploaded the track to generate viral hype.

Process and Methods

• Reactive Takedown Processes: Platforms relied on manual DMCA notices rather than proactive AI filtering.
• Workaround Metadata Use: The creator used a producer tag, enabling a temporary copyright loophole.

Regulatory Environment

• Inadequate IP Legislation: Current copyright law does not clearly protect voice prints from deepfakes.
• Unresolved Fair Use Precedent: No legal precedent exists for whether training AI on scraped data is fair use.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts of the viral song, the platforms involved, the copyright claims by UMG, and the specific reactions of the parties involved (such as Drake's Instagram post). The legal and financial implications are well-documented with specific streaming numbers and royalty estimates.
• Ambiguities identified: It is slightly unclear whether the creator pulled the track voluntarily initially to build hype or if platforms acted immediately, and whether the Metro Boomin tag was a real sample or AI-generated.
• Alternative interpretations: The incident could be viewed purely as a copyright/IP dispute rather than an AI safety issue, but the core of the dispute relies on the capabilities of generative AI to clone human voices.

An anonymous creator used generative AI voice-cloning to release a viral song mimicking artists Drake and The Weeknd. The incident triggered copyright takedowns and sparked debates on intellectual property, but presents negligible national security implications across all assessed categories.

• Overall national security impact: Negligible
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Unknown
• Alleged perpetrator: [HANDLE_001]

Threat characteristics

• Imminence: Long-term. Represents a long-term strategic and legal challenge regarding intellectual property rather than an active security crisis.
• Autonomy: Human-controlled. The AI system acted purely as a tool under the direct control of a human creator.
• Novelty: Evolved capability. Represents a significant advancement in the commercial realism and distribution of synthetic voice cloning.

Impact by dimension

• Physical security: Negligible. No physical systems, infrastructure, or human safety were impacted by this commercial music incident.
• Information security: Negligible. The incident involves commercial music deepfakes of pop artists, posing no threat to intelligence capabilities or state-sponsored information warfare.
• Sovereignty: Negligible. No impact on government functions, state authority, or electoral systems.
• Economic security: Negligible. While highlighting intellectual property and copyright challenges in the music industry, the financial impact was negligible ($1,888) and posed no threat to national economic security.
• Societal stability: Negligible. No threat to societal stability, civil liberties, or population safety.