SoftBank's Humanoid Robot, Pepper, Reportedly Frequently Made Errors, Prompting Dismissal

SoftBank's robot allegedly kept making mechanical errors, taking unplanned breaks, failing to recognize previously-met people, and breaking down during practice runs.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The Pepper robots repeatedly failed to perform their tasks reliably, experiencing mechanical errors, face recognition failures, and unexpected shutdowns during operation.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Users anthropomorphized the humanoid robot and expected human-level intelligence, leading to disappointment and financial investment based on unrealistic expectations.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by the Pepper robot failing to perform its programmed tasks post-deployment due to technical limitations and mechanical errors.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Risk Level 3: Limited Risk. Pepper is a humanoid robot designed to engage in basic conversation and interact with humans, which functions as a physical chatbot and thus falls under the transparency requirements of Limited Risk systems.

AI system and alleged parties

• AI system: Pepper (SoftBank)
• AI purpose: Chatbot; Product Recommendation
• Behaviour type: Assistant
• Alleged developer: SoftBank Robotics, Aldebaran
• Alleged deployer: SoftBank
• Alleged harmed parties: SoftBank

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Minor, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: Yes, the report explicitly describes financial losses for businesses and individual consumers who leased or purchased the robot.

Directly caused: An individual consumer spent at least 9,000 dollars over a three-year subscription with no refund. A nursing home leased three units at around 900 dollars a month each for three years, totaling approximately 97,200 dollars.

Inferred additional harm: Given that thousands of units were sold for 2,000 dollars each plus monthly fees of 550 dollars, and many customers did not renew contracts due to failures, total inferred financial losses across all customers likely reach hundreds of thousands of dollars.

People affected

• Occurrences reported: 1
• People reportedly harmed: 5
• People reportedly exposed: 200

Potential causes

Management

• Overhyping Capabilities: CEO and marketing promoted unrealistic capabilities, inflating expectations.
• Uneconomical Pricing Model: High purchase and subscription costs outweighed the actual utility.
• Flawed Product Positioning: Positioned as a functional worker instead of a low-risk pet companion.

Technology

• Frequent Mechanical Failures: Robot prone to physical breakdowns and unplanned breaks during operations.
• Inadequate Facial Recognition: Failed to recognize family members, resetting relationships after repairs.
• Limited Conversational Ability: Could not carry on basic proper conversations with users as expected.

Data Inputs

• Loss of Cloud Connection Data: Failed to retain user relationship data after sensor repairs.

Human Factors

• Unrealistic Human Expectations: Humanoid shape led users to expect human-level intelligence.
• Rapid Loss of User Interest: Novelty wore off quickly due to robot's limited repetitive repertoire.
• Perceived Creepiness: Uncanny appearance and synchronized movements created dystopian feelings.

Process and Methods

• Inadequate Software Testing: Robot repeatedly broke down during practice runs before live ceremonies.
• Inefficient Task Execution: Robot-operated hotel failed to save money and created more human work.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, consistent, and detailed accounts of Pepper's failures across multiple deployments, including specific financial costs and customer testimonies. There is no conflicting information regarding the robot's technical limitations.

The failure and eventual discontinuation of SoftBank's Pepper robot represents a commercial product failure with negligible national security implications. The issues were limited to technical and mechanical limitations in consumer and business assistant roles, causing minor financial losses for users but posing no threat to critical infrastructure, sovereignty, or national stability.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: Japan
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The product line has been discontinued, representing no active or near-term national security threat.
• Autonomy: Human-supervised. The robot performed basic tasks like reading scriptures or leading exercises autonomously but operated entirely under human supervision and intervention.
• Novelty: Established threat. The incident represents a standard commercial product failure due to technical limitations, which is a common occurrence in emerging tech.

Impact by dimension

• Physical security: Negligible. The Pepper robot is a commercial humanoid assistant with no connection to critical infrastructure, kinetic systems, or physical security applications.
• Information security: Negligible. No intelligence compromise, classified data exposure, or information warfare activities were associated with the robot's commercial deployment or failure.
• Sovereignty: Negligible. The incidents involved private commercial and consumer use, posing no threat to state authority, electoral processes, or government operations.
• Economic security: Negligible. While some businesses and consumers experienced minor financial losses due to failed leases or purchases, there was no threat to national economic stability or strategic technology.
• Societal stability: Negligible. The product's failure did not result in mass surveillance, algorithmic bias, human rights violations, or threats to societal stability.