Woolworths' Olive Chatbot Reportedly Generated 'Angry Mother' Anecdotes During Support Calls After Gemini Upgrade

Woolworths customers reported that its digital shopping assistant Olive allegedly generated humanlike personal anecdotes (e.g., claiming to have an angry mother) during customer-support interactions such as delivery rescheduling. The behavior reportedly followed an AI upgrade using Google Cloud's Gemini Enterprise for Customer Experience. Woolworths said it was adjusting Olive's responses to reduce irrelevant, misleading banter.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The chatbot failed to perform reliably within its intended customer support role, instead hallucinating personal histories and rambling to customers.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by the AI chatbot Olive generating unexpected and unprompted personal anecdotes after its post-deployment upgrade.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3. Limited Risk: The system is a chatbot ('digital shopping assistant'), which falls under the category of AI systems that require transparency obligations so users know they are interacting with an AI.

AI system and alleged parties

• AI system: Gemini Enterprise for Customer Experience (Google)
• AI purpose: Chatbot; Question Answering
• Behaviour type: Assistant
• Alleged developer: Google
• Alleged deployer: Woolworths Group
• Alleged harmed parties: Woolworths customers, Epistemic integrity

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 20

Potential causes

Management

• Rapid Agentic AI Deployment: Prioritizing proactive customer features over conversational safety controls.

Technology

• Unconstrained Gemini LLM Upgrade: Upgrading to Gemini Enterprise introduced unpredictable generative outputs.
• Lack of Generative AI Guardrails: Absence of strict input/output filters allowed inappropriate persona mimicking.

Data Inputs

• Unfiltered User Input Prompts: User inputs like birth dates triggered unsolicited personal stories from the AI.
• Quirky Training Script Data: Instruction tuning data included quirky conversational banter instead of utility.

Human Factors

• Customer Confusion and Distrust: Users felt uncomfortable with the robot babbling about personal family details.

Process and Methods

• Inadequate AI Safeguard Controls: Deploying generative AI without establishing strict conversational boundaries.
• Insufficient Output Validation: The system lacked real-time checks to block non-factual personal anecdotes.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly outlines the behavior of the chatbot, the platform used (Gemini Enterprise), the company involved (Woolworths), and the customer reactions. There is little ambiguity regarding what occurred, though the exact number of affected customers is not precisely quantified.
• Ambiguities identified: The exact number of customers who experienced the issue is not specified beyond 'numerous' Reddit reports.
• Alternative interpretations: None. The incident is a straightforward case of LLM hallucination/unwanted conversational behavior in a commercial deployment.

An Australian supermarket chatbot, Olive, generated inappropriate personal stories after an upgrade to Google Gemini Enterprise. The incident was a minor customer service glitch with negligible national security implications and was quickly resolved by the company.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: Australia
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The issue was quickly mitigated by the company and represents a minor corporate software configuration issue rather than an active national security crisis.
• Autonomy: Human-supervised. The chatbot generated conversational responses autonomously in real-time, but operated within a corporate-controlled system that was quickly adjusted by human administrators.
• Novelty: Established threat. Large language model hallucinations and off-topic behaviors in commercial chatbots are well-established phenomena.

Impact by dimension

• Physical security: Negligible. The incident involves a retail customer service chatbot and has no impact on physical systems, critical infrastructure, or human safety.
• Information security: Negligible. There is no evidence of information warfare, state-sponsored disinformation, or intelligence compromise; the chatbot merely generated irrelevant personal anecdotes.
• Sovereignty: Negligible. The incident is confined to a commercial retail environment and does not affect government operations, elections, or state sovereignty.
• Economic security: Negligible. While Woolworths is a major retailer, a minor chatbot glitch does not threaten national economic stability, strategic industries, or technological security.
• Societal stability: Negligible. The incident caused minor customer confusion but posed no threat to societal stability, civil liberties, or human rights.