McDonald's Reportedly Ends IBM Partnership After AI Drive-Thru Ordering Errors at U.S. Locations

Between 2021 and mid-2024, McDonald's reportedly piloted an AI-enabled voice ordering system that was developed with IBM and based on its 2019 acquisition of Apprente. It was reportedly deployed at at over 100 U.S. drive-thrus. Social media posts alleged frequent misorders, such as adding unwanted items, mixing adjacent lane orders, and ignoring corrections. In June 2024, McDonald's confirmed it ended the IBM pilot, citing plans to explore alternative voice-AI vendors.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The AI system failed to perform reliably under varying conditions, such as handling crosstalk from adjacent lanes or understanding standard customer requests, resulting in incorrect orders.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by the deployed IBM AI voice ordering system failing to accurately process customer orders in real-world drive-thru environments.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system is an AI-powered voice chatbot used for automated order taking, which falls under the category of chatbots requiring transparency so users know they are interacting with an AI.

AI system and alleged parties

• AI system: Automated Order Taking (AOT) (IBM)
• AI purpose: AI Voice Assistant; Voice Recognition
• Behaviour type: Autonomous
• Alleged developer: IBM
• Alleged deployer: McDonald's
• Alleged harmed parties: McDonald's customers

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 1
• People reportedly harmed: 4
• People reportedly exposed: 4

Potential causes

Management

• Premature System Deployment: Trialing AI in live drive-thrus before achieving target 95 percent accuracy.
• High Upfront and Support Costs: High maintenance and integration costs reduced economic viability of the AI.

Technology

• Brittle Tree-Based NLU Model: IBM model used rigid logic trees instead of flexible LLMs to parse orders.
• Poor Voice Recognition Accuracy: Accuracy was in the low 80s, failing the required 95 percent threshold.
• Inadequate Noise Filtering: System could not filter out crosstalk from adjacent drive-thru lanes.

Data Inputs

• Crosstalk Interference: Audio from neighboring lanes was processed as the primary customer's input.
• Out of Domain Requests: Customer requests for non-menu items like Mountain Dew caused system errors.

Human Factors

• Mid-Order Customer Changes: Customers changing their minds broke the rigid decision-tree logic of the AI.
• Customer Frustration: Screaming or laughing customers further degraded voice recognition performance.

Process and Methods

• Rigid Error Correction Flow: Error correction attempts triggered compounding mistakes like adding extra items.
• Delayed Human Intervention: System failed to hand off to human staff quickly during cascading errors.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts of the pilot program, the specific errors made by the AI system, and the subsequent decision by McDonald's to terminate the partnership with IBM. Multiple sources corroborate the accuracy rates and the nature of the customer complaints.

A pilot program for an AI voice-ordering system at McDonald's drive-thrus was terminated due to high error rates and customer frustration. The incident is purely commercial and technical in nature, carrying negligible national security implications across all assessed categories.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The incident is a concluded commercial pilot with no active or emerging national security threat vector.
• Autonomy: Human-supervised. The AI operated autonomously to take orders but was supervised by human restaurant staff who corrected errors or took over when failures occurred.
• Novelty: Established threat. Speech recognition and natural language processing failures in noisy environments are well-documented and common technical limitations.

Impact by dimension

• Physical security: Negligible. The incident involves a commercial fast-food ordering system failure with no impact on critical infrastructure, physical systems, or human safety.
• Information security: Negligible. No classified systems, intelligence assets, or information operations were involved or affected by the drive-thru voice system.
• Sovereignty: Negligible. The failure of a commercial drive-thru ordering system does not impact government operations, state authority, or sovereign functions.
• Economic security: Negligible. While representing a minor commercial setback for McDonald's and IBM, the incident has no bearing on strategic national economic or technological security.
• Societal stability: Negligible. The incident caused only minor customer frustration and humorous social media posts, presenting no threat to societal stability or civil liberties.