Anysphere AI Support Bot for Cursor Reportedly Invents Login Policy, Leading to Subscription Cancellations

In April 2025, users of Cursor, an AI-powered coding assistant developed by Anysphere, reported being logged out unexpectedly. An AI-powered support bot, "Sam," allegedly responded with an invented login policy to justify the behavior. The hallucinated policy was not based on any real company change. The incident reportedly led to subscription cancellations.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The AI support bot failed to perform reliably by hallucinating a non-existent company policy and presenting it as fact to customers.

Additional risk subdomains

• 3.1 False or misleading information: The bot generated and spread false information about company policies, leading to inaccurate beliefs and subscription cancellations.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by the Cursor support AI bot hallucinating a non-existent policy, which was an unexpected and unintentional outcome after its deployment.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system is a customer support chatbot. The report notes that the company 'failed to tell users that its customer support person is actually a hallucinating bot', highlighting transparency issues central to Limited Risk systems.

AI system and alleged parties

• AI system: Cursor AI support bot (Anysphere)
• AI purpose: Chatbot; Question Answering
• Behaviour type: Assistant
• Alleged developer: Unknown LLM developer, Anysphere
• Alleged deployer: Cursor, Anysphere
• Alleged harmed parties: Software developers, Small businesses relying on Cursor, Cursor users

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 1
• People reportedly harmed: 5
• People reportedly exposed: 10

Potential causes

Management

• Failure to Disclose AI Use: Management failed to inform users that support was handled by a bot.
• Over-Reliance on Automation: Deploying autonomous bots to save labor costs without risk mitigation.

Technology

• AI Chatbot Hallucinations: The bot confidently made up a non-existent login policy to explain a bug.
• Compounding Errors in Reasoning: Step-by-step reasoning models can compound errors at each thinking step.
• Forgetting Tasks via RL: Reinforcement learning focus on some tasks makes the AI forget others.

Data Inputs

• Lack of Grounded Truth Data: AI models cannot determine objective truth from their training data.
• Training on Uncurated Web Data: Exhausting English internet text led to reliance on unverified source data.

Human Factors

• User Deception by Machine: Users felt tricked because the bot posed as a human support agent.
• Lack of Human Oversight: No human reviewed the bot's email before it was sent to customers.

Process and Methods

• Absence of Guardrails: No constraints were placed on the bot to prevent unsupported claims.
• Repetitive AI Canned Responses: Support relied on repetitive AI responses instead of resolving issues.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, consistent details about the specific incident involving Cursor's support bot, including the nature of the hallucination, the company's response, and the user reaction. There is minimal ambiguity regarding the core events.
• Ambiguities identified: The exact number of users who received the email or canceled their subscriptions is not specified, referred to only as 'several' or 'some'.

An AI customer support bot for the coding assistant Cursor hallucinated a non-existent login policy, causing user confusion and minor financial loss for the company through subscription cancellations. The incident has negligible national security implications and represents a standard commercial AI reliability issue.

• Overall national security impact: Negligible
• Scope: Unknown
• Primary target: No clear primary
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. This is a resolved customer service incident with no ongoing threat to national security.
• Autonomy: Full autonomy. The AI support bot operated autonomously to generate and send emails to customers without human review or intervention.
• Novelty: Established threat. AI hallucinations and errors in automated customer-facing systems are well-documented and common occurrences.

Impact by dimension

• Physical security: Negligible. No threat to physical systems, infrastructure, or human safety was reported in connection with this customer support bot failure.
• Information security: Negligible. No intelligence compromise or systematic information warfare campaign; the incident was an isolated, unintentional hallucination by a commercial support bot.
• Sovereignty: Negligible. No impact on government operations, state authority, or electoral systems.
• Economic security: Negligible. The incident caused minor financial losses and cancellations for a private software startup, representing no threat to strategic economic or technological security.
• Societal stability: Negligible. The incident involved standard customer service confusion and has no implications for societal stability, civil liberties, or human rights.