Bing Chat's Outputs Featured in Demo Video Allegedly Contained False Information

Microsoft's demo video of Bing Chat reportedly featured false or made up information such as non-existent pet vacuums features or false figures on financial statements.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The Bing Chat AI system generated and spread incorrect, fabricated, and misleading financial and product information during a highly publicized demonstration.

Additional risk subdomains

• 7.3 Lack of capability or robustness: The model demonstrated a clear lack of capability and robustness in accurately extracting and summarizing structured financial data from documents.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by the Bing Chat AI system which unintentionally generated incorrect and hallucinated information while attempting to fulfill user search and summarization queries.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system is a general-purpose conversational chatbot (Bing Chat) which falls under the Limited Risk category, requiring transparency obligations so users know they are interacting with an AI.

AI system and alleged parties

• AI system: Bing Chat (Microsoft)
• AI purpose: Chatbot; Question Answering
• Behaviour type: Assistant
• Alleged developer: OpenAI, Microsoft
• Alleged deployer: Microsoft
• Alleged harmed parties: Microsoft

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Epistemic

Reported: The report explicitly describes epistemic harm in the form of AI-driven fabrication, hallucination, and incorrect financial and product data presented as facts.

Directly caused: The Bing AI system generated completely fabricated financial statistics for Gap and Lululemon, made up product specifications for a Bissell vacuum, and hallucinated details about Mexico City nightlife during a public demo.

Indirectly caused: The dissemination of these errors led to a false public narrative and misinformation about the capabilities of the AI system and the financial health of the mentioned companies.

Inferred additional harm: Users relying on these search results for real-world decision-making (e.g., purchasing vacuums, planning trips, or making investment decisions) would be misled by false information.

People affected

• Occurrences reported: 1
• People reportedly exposed: 10000

Potential causes

Management

• Prioritizing Speed Over Safety: Rushing to launch and demo to compete with rivals led to missed errors.
• Lack of Demo Accountability: No clear ownership for verifying the truthfulness of public demo content.

Technology

• AI Hallucination of Facts: The model confidently generated false details about nightlife and vacuums.
• Inaccurate Data Extraction: The model failed to extract correct numbers from financial PDF documents.
• Product Version Confusion: The model confused corded and cordless product specs despite citation.

Data Inputs

• Source Citation Discrepancy: Model claims to use specific sources but ignores or distorts their content.
• Missing Real-Time Ratings: Lack of integration with current web reviews led to claims of no ratings.
• Ambiguous Context Matching: Difficulty distinguishing between similarly named products in data sources.

Human Factors

• Lack of Pre-Demo Verification: The Bing demo team failed to verify the accuracy of the generated outputs.
• Overconfidence in AI Output: Presenters assumed AI-generated financial summaries were correct.

Process and Methods

• Inadequate Demo QA Process: No rigorous review process was applied to pre-recorded demo outputs.
• Flawed Evaluation Metrics: Evaluation methods failed to catch hallucinated financial metrics.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides highly detailed, verifiable examples of the AI's errors during the demo, comparing the AI's outputs directly with the source documents (HGTV article, Gap and Lululemon financial statements). The evidence of hallucination is clear and unambiguous.
• Ambiguities identified: None significant; the discrepancies between the AI's claims and the source texts are factual and clearly demonstrated.
• Alternative interpretations: None; the errors are clear mathematical and factual hallucinations.

A public demonstration of Microsoft Bing Chat featured factual errors and financial data hallucinations. The incident highlights known technical limitations of large language models in data extraction and factual accuracy, but carries negligible national security implications.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: No clear primary
• Alleged perpetrator: Microsoft

Threat characteristics

• Imminence: Long-term. The incident represents a technical limitation of commercial LLMs rather than an active national security crisis.
• Autonomy: Human-supervised. The AI generated the responses autonomously based on user prompts, but operates within a consumer-facing assistant framework.
• Novelty: Established threat. Large language model hallucinations and data extraction failures are well-documented technical limitations.

Impact by dimension

• Physical security: Negligible. The incident involved text generation errors in a commercial search chatbot demo and posed no threat to physical systems, critical infrastructure, or human safety.
• Information security: Negligible. The factual errors were unintentional hallucinations during a product demonstration, not a coordinated disinformation campaign or intelligence compromise.
• Sovereignty: Negligible. No core government operations, electoral systems, or sovereign decision-making processes were affected by the chatbot errors.
• Economic security: Negligible. While the system fabricated financial data for private retail companies, it did not target critical financial infrastructure or threaten national economic security.
• Societal stability: Negligible. The incident did not involve mass surveillance, systematic discrimination, or threat to civil liberties and population safety.