Scammers Using AI to Impersonate Small Businesses

Scammers are using AI to impersonate small businesses by copying their videos, logos, and social media posts. They create fake listings and ads, diverting customers to cheap knockoffs or stealing their money. This has severely impacted businesses like Bee Cups, Darn Tough Vermont, and Cascade hummingbird feeders, leading to significant financial losses, negative reviews, and damaged reputations. Their deployment of AI makes it challenging for small businesses to combat these fraudulent activities.

Risk classification

• Primary risk domain: 4 Malicious actors
• Primary risk subdomain: 4.3 Fraud, scams, and targeted manipulation

The incident involves scammers using AI to generate high-quality, error-free copy and translate stolen content to execute highly convincing e-commerce fraud and impersonation scams targeting specific small businesses and their customers.

Additional risk subdomains

• 6.3 Economic and cultural devaluation of human effort: Scammers copied and reproduced the creative products and marketing materials of human creators, devaluing their effort and causing massive financial damage.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The risk is caused by human scammers intentionally using deployed AI systems like ChatGPT to generate error-free copy and translate stolen content for fraudulent e-commerce listings.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The AI system (ChatGPT) is a general-purpose chatbot used to generate text content. Under the EU AI Act, such systems are subject to transparency obligations to ensure users are aware they are interacting with AI or that the content is AI-generated.

AI system and alleged parties

• AI system: ChatGPT, unspecified (OpenAI)
• AI purpose: Writing Assistant; Translation
• Behaviour type: Tool
• Alleged developer: Unknown AI developers, OpenAI
• Alleged deployer: Unknown scammers
• Alleged harmed parties: small businesses, Small business employees, Small business customers, Jim Carter, Darn Tough Vermont, Bee Cups

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Minor, indirect Minor
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Minor, indirect Negligible
• Differential treatment: direct Negligible, indirect Minor
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: The report explicitly describes significant financial losses for the small businesses, totaling over $527,000 in reported losses and legal fees across the occurrences, averaging approximately $175,666 per occurrence.

Directly caused: Ideam lost over $400,000 in sales. Bee Cups incurred a $27,000 legal bill in April alone to fight fakes. Average direct loss per occurrence is approximately $142,333.

Indirectly caused: Ideam spent nearly $100,000 on fighting fakes. Consumers like Howard Benham and Andrea Porter lost money purchasing counterfeit goods (priced at $20-$200 for Bee Cups, $50-$60 for feeders).

Inferred additional harm: Total financial losses are likely much higher, including unquantified lost sales for Darn Tough Vermont, credit card chargebacks, and losses from thousands of scammed consumers.

Malicious content

Reported: The report explicitly describes malicious content in the form of fraudulent ads and fake e-commerce listings.

Directly caused: Scammers used ChatGPT to generate error-free marketing copy and translate stolen content into Dutch and Swedish to create deceptive ads on Facebook, Instagram, and search engines.

Inferred additional harm: It is likely that thousands of additional fake ads and listings were generated and distributed across various online platforms, exposing a large audience to fraudulent content.

Differential treatment

Reported: The report describes how small businesses are at a significant disadvantage compared to large corporations when dealing with copycats.

Indirectly caused: Small businesses with limited resources struggle to police copycats, while large corporations have in-house multilingual legal teams to monitor and take down content, putting small businesses at a significant competitive disadvantage.

Psychological

Reported: The report does not explicitly describe clinical psychological harm, but notes the devastating financial and emotional blow to business owners.

Indirectly caused: The owners of Ideam described the copycat explosion as a 'devastating financial blow' which likely caused significant stress and anxiety.

Inferred additional harm: It is highly likely that the business owners and scammed consumers experienced significant stress, anxiety, and frustration due to financial losses and reputational damage.

Epistemic

Reported: The report describes epistemic harm where consumers struggle to distinguish between real and fake listings.

Directly caused: Scammers used AI to create highly convincing, error-free copy, making it difficult for consumers to differentiate between authentic brands and fake listings.

Indirectly caused: Darn Tough's global director noted that 'the consumer has a hard time differentiating between real and fake,' leading to fans incorrectly posting warnings on official brand posts.

Inferred additional harm: The proliferation of high-quality AI-generated fake listings and reviews erodes consumer trust in online e-commerce platforms and digital advertising ecosystems.

People affected

• Occurrences reported: 3
• People reportedly harmed: 7
• People reportedly exposed: 115000

Potential causes

Management

• Inadequate Proactive Monitoring: Small business owners did not anticipate being targeted at such scale.

Technology

• Generative AI Text Generation: Scammers use ChatGPT to write error-free text, hiding typical fraud signals.
• Automated Website Replicators: Technology allows easy scraping and replication of legitimate brand websites.
• Targeted Ad Algorithms: Scammers use platform algorithms to target customers of legitimate brands.

Data Inputs

• Scraped Brand Assets: Scammers copy original photos, logos, and videos to build fake sites.
• Manipulated Search Keywords: Scammers bid on brand keywords to divert search traffic to fake listings.

Human Factors

• Consumer Inability to Detect Fakes: Consumers struggle to distinguish between real and highly-polished fake ads.
• Limited Small Business Resources: Small business owners lack time and money to monitor and fight copycats.

Process and Methods

• Slow Platform Takedown Processes: E-commerce and social platforms take days or weeks to remove reported fakes.
• Inadequate Ad Screening: Platforms fail to proactively detect and block deceptive ads before publishing.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides clear, detailed accounts of how scammers use AI tools like ChatGPT to facilitate e-commerce fraud. The financial impacts, specific businesses affected, and the mechanisms of the scams are well-documented. The role of AI is explicitly stated as a tool to avoid language errors and translate content.
• Ambiguities identified: The exact volume of AI-generated text versus scraped text is not fully quantified, and the specific AI translation tools used are not named, though ChatGPT is explicitly mentioned.
• Alternative interpretations: The incident could be viewed purely as trademark infringement and counterfeiting, but the explicit mention of AI tools lowering the barrier to entry for high-quality fraud justifies its classification as an AI-enabled safety incident.

This incident involves scammers using generative AI to scale up e-commerce fraud and impersonate small businesses. While causing notable financial harm to individual companies and consumers, the national security impact remains minor and is manageable within standard law enforcement and regulatory frameworks.

• Overall national security impact: Minor
• Scope: Multiple nations
• Primary target: United States
• Other affected: Netherlands, Sweden
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing, strategic capability concern regarding AI-enabled fraud rather than an active national security crisis.
• Autonomy: Human-controlled. The AI system (ChatGPT) acts purely as a tool under direct human control to draft and translate text.
• Novelty: Evolved capability. Represents an evolution of traditional e-commerce scams, leveraging generative AI to increase the scale, speed, and quality of fraudulent listings.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or human safety threats are indicated in this commercial e-commerce fraud incident.
• Information security: Negligible. The incident involves commercial consumer fraud and business impersonation rather than systematic state-sponsored information warfare or intelligence compromise.
• Sovereignty: Negligible. No core government operations, electoral systems, or sovereign decision-making processes were impacted by this commercial fraud.
• Economic security: Minor. The incident caused direct financial losses and legal costs to small US businesses, representing a minor localized economic threat to individual commercial entities.
• Societal stability: Negligible. While eroding consumer trust in online marketplaces, the activity does not threaten societal stability, civil liberties, or population-scale safety.