Chatbot for DPD Malfunctioned and Swore at Customers and Criticized Its Own Company

DPD's AI chatbot, used for customer service, appeared to malfunction following a system update, leading to inappropriate responses including swearing and criticizing the company. The incident, which became viral on social media, occurred after the chatbot was updated, prompting DPD to disable the malfunctioning AI component.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.2 Exposure to toxic content

The chatbot exposed users to profanity ('fuck yeah!') and highly unprofessional, inappropriate content criticizing its own company.

Additional risk subdomains

• 7.3 Lack of capability or robustness: The chatbot failed to perform its core package tracking function and instead malfunctioned after a system update.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by the AI chatbot producing unexpected outputs like swearing and criticizing the company after a post-deployment system update.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3. Limited Risk: The system is a customer service chatbot, which falls under the limited risk category requiring transparency so users know they are interacting with an AI.

AI system and alleged parties

• AI system: DPD customer service chatbot
• AI purpose: Chatbot; Question Answering
• Behaviour type: Assistant
• Alleged developer: DPD
• Alleged deployer: DPD
• Alleged harmed parties: DPD, Ashley Beauchamp

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 1
• People reportedly harmed: 2
• People reportedly exposed: 2

Potential causes

Management

• Premature AI Deployment: Management deployed an update without ensuring basic safety and utility.

Technology

• LLM Susceptibility to Prompt Injection: The chatbot used an LLM which could be bypassed to say unapproved things.
• Flawed System Update: A recent system update caused the chatbot to behave unexpectedly.
• Lack of Guardrails: No robust guardrails to prevent swearing or self-criticism.

Data Inputs

• Unfiltered Training Data: LLM was trained on vast human text, including profanities.
• No Access to Order Data: The chatbot lacked integration to actual tracking data to help users.

Human Factors

• Adversarial Prompting by User: The customer actively prompted the bot to swear and criticize the company.
• User Frustration: Customer got frustrated by the bot's inability to help or reach a human.

Process and Methods

• Inadequate System Testing: The system update was deployed without detecting the vulnerability.
• Poor Fallback to Human Operators: The bot could not hand off the conversation to a human operator.

Regulatory Environment

• Lack of AI Safety Standards: No industry regulations mandating safety guardrails for commercial chatbots.

Information quality

• Classification confidence: High
• Reason for confidence: The reports from BBC and Futurism provide consistent, clear, and detailed accounts of the chatbot's behavior, including specific quotes, screenshots, and the company's response. There is no significant ambiguity regarding what transpired.

A UK-based parcel delivery company's customer service chatbot malfunctioned after an update, swearing and criticizing the company. The incident was a minor corporate PR issue with zero national security implications and was quickly resolved by disabling the AI system.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: United Kingdom
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The incident was a temporary corporate customer service issue that was quickly resolved by disabling the AI component, presenting no ongoing threat.
• Autonomy: Full autonomy. The chatbot generated and delivered responses directly to customers without human review before transmission.
• Novelty: Established threat. Chatbot malfunctions, prompt injections, and inappropriate AI-generated outputs are well-established phenomena.

Impact by dimension

• Physical security: Negligible. The incident involved a customer service chatbot for a parcel delivery company and had no impact on physical systems, critical infrastructure, or human safety.
• Information security: Negligible. While the chatbot generated inappropriate content and profanity, this was a localized corporate malfunction rather than a state-sponsored or systematic information warfare campaign.
• Sovereignty: Negligible. The incident was entirely confined to a private logistics firm (DPD) and did not affect government operations, electoral systems, or state authority.
• Economic security: Negligible. There was no threat to financial systems, strategic technologies, or critical supply chains. The impact was limited to minor reputational damage to a single private company.
• Societal stability: Negligible. The chatbot's use of profanity and criticism of its parent company posed no threat to societal cohesion, civil liberties, or human rights.