Alleged ChatGPT-Generated Book with a Duplicate Title, Fake Author, and Similar Content Surfaces on Amazon Ahead of Real Author's Book Release

The author Chris Cowell had spent more than a year writing his book "Automating DevOps with GitLab CI/CD Pipelines" when, three weeks before its release, another book appeared bearing the exact title by an author (Marie Karpos) for whom no information could be found. The book appeared to have been written by ChatGPT. While the original Washington Post story does not say so, it is possible the name and description were taken from the Amazon preorder page.

Risk classification

• Primary risk domain: 6 Socioeconomic & Environmental
• Primary risk subdomain: 6.3 Economic and cultural devaluation of human effort

The incident highlights how generative AI is used to copy and devalue human creative and technical writing, leading to a chilling effect on human authors who face automated competition.

Additional risk subdomains

• 4.3 Fraud, scams, and targeted manipulation: Publishers used fake author profiles and copycat book titles to siphoning sales and gain an illegitimate financial benefit.
• 3.2 Pollution of information ecosystem and loss of consensus reality: AI-generated content is flooding search engines, product reviews, and news sites, polluting the information ecosystem with unvetted and fabricated material.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The risk is caused by human publishers intentionally deploying generative AI tools post-deployment to rapidly produce copycat books and low-quality web content for financial gain.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The report describes the use of chatbots (ChatGPT) and generative AI to produce text and books, which falls under the category of AI-generated content requiring transparency obligations.

AI system and alleged parties

• AI system: ChatGPT (OpenAI)
• AI purpose: Writing Assistant; Technical Text Generation
• Behaviour type: Assistant
• Alleged developer: OpenAI, ChatGPT
• Alleged deployer: Marie Karpos, InKstall
• Alleged harmed parties: Chris Cowell

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Minor
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Substantial
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: The report describes potential financial losses from siphoned book sales and the displacement of human writers, but does not explicitly quantify the exact losses.

Directly caused: The author suffered unquantified direct financial losses due to the copycat book siphoning potential pre-order sales on Amazon before it was removed.

Indirectly caused: Human content platforms like Textbroker reported slowed growth as clients experimented with cheaper, automated AI content instead of hiring human writers.

Inferred additional harm: We can infer that the author and other copied creators suffered hundreds to thousands of dollars in lost royalties, and freelance writers lost significant income due to automated content mills.

Epistemic

Reported: The report explicitly describes epistemic harm, including AI-driven misinformation, fabrication, and the erosion of shared truth.

Directly caused: NewsGuard identified 49 AI-generated news sites publishing fabricated stories, such as a false report claiming Joe Biden had died, and incorrect celebrity profiles generated by Ingenio's AI tools.

Indirectly caused: The proliferation of unvetted AI-generated articles and product reviews on search engines and Amazon makes it difficult for consumers to identify factual reality.

Inferred additional harm: Millions of internet users are likely misled by fabricated news stories and AI-generated content ranking highly on search engines, leading to a broader erosion of public trust in online information.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 1

Potential causes

Management

• Prioritizing Volume Over Quality: Companies focus on high-volume clickbait to maximize ad revenue.
• Inadequate Risk Assessment: Publishers adopt AI without assessing reputational risks or errors.
• Workforce Restructuring: Media companies lay off staff while shifting to AI workflows.

Technology

• AI Propensity to Hallucinate: Models generate false facts, like claiming a retired pitcher is a partner.
• Lack of Grounding in Reality: Generative tools create unvetted prose that lacks factual grounding.
• Plagiarism and Copycatting: AI generates books mirroring specific titles and content of human authors.

Data Inputs

• Unvetted Online Training Data: Models train on vast, unvetted data pools, losing track of truth.
• Inauthentic Platform Reviews: Fake reviews are used to boost visibility of low-quality AI books.

Human Factors

• Overreliance on Automated Outputs: Publishers post AI text directly without reviewing or editing it.
• Economic Incentives for Cheap Content: Creators prioritize low cost over content quality and accuracy.
• Chilling Effect on Human Authors: Human writers are discouraged by cheap, automated competition.

Process and Methods

• Absence of Human-in-the-Loop Review: Direct publishing of AI outputs without quality control checks.
• Flawed Platform Filtering: Platforms fail to detect copycat titles and fake reviews effectively.
• Poor Disclosure Practices: Sites hide AI use or make disclosures obscure to readers.

Regulatory Environment

• Lack of AI Liability Frameworks: Uncertainty around AI liability and intellectual property rules.
• Permissive Platform Search Rules: Search engines allow AI content if it does not gamify rankings.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides clear, detailed, and verified accounts of the copycat book incident on Amazon, backed by interviews with the affected author, industry experts, and references to investigations by NewsGuard.
• Ambiguities identified: The exact financial impact on the author is not quantified, and the precise AI model used by the copycat publisher is inferred rather than explicitly confirmed, though highly likely to be ChatGPT.
• Alternative interpretations: The incident could be viewed primarily as a copyright/IP infringement issue rather than an AI safety issue, but the central role of generative AI in enabling this scale of copycat publishing makes it an AI safety risk.

The incident involves the commercial use of generative AI to rapidly produce copycat books and fake news websites. While it highlights growing challenges in intellectual property protection and digital information integrity, its national security impact remains minor, with no direct threats to critical infrastructure, sovereignty, or national security capabilities.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: United States, India
• Alleged perpetrator: Commercial publishers

Threat characteristics

• Imminence: Long-term. Represents an ongoing, long-term strategic challenge regarding digital information integrity and intellectual property protection rather than an immediate national security crisis.
• Autonomy: Human-supervised. AI models generate content autonomously based on initial human prompts, with varying levels of human oversight and editing before publication.
• Novelty: Evolved capability. Plagiarism and fake news are established threats, but the use of generative AI represents a significant advancement in the scale, speed, and low cost of content generation.

Impact by dimension

• Physical security: Negligible. No threats to physical systems, critical infrastructure, or human safety are indicated in the incident details.
• Information security: Minor. The proliferation of AI-generated fake news sites publishing fabricated stories represents a minor information integrity concern, though currently driven by commercial spam rather than coordinated state adversaries.
• Sovereignty: Negligible. No threats to state authority, electoral systems, or core government decision-making processes are demonstrated.
• Economic security: Minor. Represents minor economic friction and intellectual property concerns for individual creators and self-publishing platforms, manageable through standard legal and platform content moderation procedures.
• Societal stability: Minor. The proliferation of AI-generated content and fabricated news slightly erodes public trust in online information but does not currently threaten large-scale societal stability or human rights.