OpenAI's Training Data for LLMs Allegedly Comprised of Copyrighted Books

Two authors alleged in a class action lawsuit OpenAI infringed authors' copyrights by incorporating illegal "shadow libraries" offering copyrighted books without permission in the training data of its generative LLMs, such as ChatGPT.

Risk classification

• Primary risk domain: 6 Socioeconomic & Environmental
• Primary risk subdomain: 6.3 Economic and cultural devaluation of human effort

The unauthorized training of ChatGPT on copyrighted books devalues the creative effort and intellectual property of authors by reproducing their work without compensation.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Pre-deployment

The risk of copyright infringement was caused by the intentional decision of humans at OpenAI to train ChatGPT on copyrighted books before the model was deployed.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system is described as ChatGPT, which is a chatbot. Under the EU AI Act, chatbots are classified as Limited Risk (Level 3) and are subject to specific transparency obligations.

AI system and alleged parties

• AI system: ChatGPT (OpenAI)
• AI purpose: Writing Assistant; Question Answering
• Behaviour type: Assistant
• Alleged developer: OpenAI
• Alleged deployer: OpenAI
• Alleged harmed parties: Paul Tremblay, Mona Awad, authors of copyrighted works

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Substantial, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: The report explicitly describes potential financial losses due to copyright infringement, though the exact damages sought are unspecified.

Directly caused: The lawsuit seeks unspecified money damages on behalf of a nationwide class of copyright owners for the unauthorized use of their books.

Inferred additional harm: We can infer significant potential financial harm to the creative industry and authors whose books were used without compensation, potentially amounting to millions of dollars in unpaid licensing fees.

People affected

• Occurrences reported: 1
• People reportedly harmed: 2
• People reportedly exposed: 2

Potential causes

Management

• Rapid Scaling Prioritization: Focusing on rapid deployment over resolving copyright compliance.

Technology

• Generative AI Training Methods: Models require massive datasets to learn language patterns effectively.
• Accurate Summary Generation: System reconstructs detailed book content, proving ingestion of works.

Data Inputs

• Use of Shadow Libraries: Training data included illegal sources containing copyrighted books.
• Unlicensed Data Scraping: Scraping internet data without verifying copyright or getting consent.

Process and Methods

• Inadequate Data Provenance: No robust filtering to exclude copyrighted materials from training sets.
• Lack of Rights Clearance: No process to obtain explicit permission from copyright holders.

Regulatory Environment

• Ambiguous Fair Use Laws: Unclear legal boundaries regarding AI training on copyrighted works.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly outlines the legal dispute, the parties involved (OpenAI and the authors), the specific AI system (ChatGPT), and the core allegation of unauthorized training on copyrighted books. There is little ambiguity regarding the facts of the lawsuit, though the legal outcome remains pending.
• Ambiguities identified: The exact financial damages sought are unspecified, and the specific list of books included in the training data is not fully detailed beyond the estimate of 300,000 books.
• Alternative interpretations: The incident could be interpreted as a standard copyright dispute rather than an AI safety incident, but the systemic implications for generative AI training data make it highly relevant to AI governance.

Authors Paul Tremblay and Mona Awad filed a class action lawsuit against OpenAI, alleging unauthorized use of copyrighted books to train ChatGPT. While the case highlights critical unresolved legal questions around intellectual property and training data for strategic AI models, its direct national security implications are currently minor.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: OpenAI

Threat characteristics

• Imminence: Long-term. This represents an ongoing legal and regulatory debate regarding intellectual property rather than an active crisis.
• Autonomy: Human-controlled. The training data selection and model development process were entirely determined and executed by humans.
• Novelty: Evolved capability. This represents a significant continuation and evolution of legal challenges facing generative AI developers over training data.

Impact by dimension

• Physical security: Negligible. No physical security threats or critical infrastructure impacts are reported in this copyright dispute.
• Information security: Negligible. No intelligence compromise, classified data theft, or information warfare operations are indicated.
• Sovereignty: Negligible. The incident involves a civil legal dispute between private parties and does not impact sovereign government operations.
• Economic security: Minor. The lawsuit highlights intellectual property risks in AI training, which could affect the business models and competitive landscape of domestic AI developers.
• Societal stability: Negligible. No threats to societal stability, civil liberties, or population safety are present in this intellectual property dispute.