AI Chatbot Reportedly Recommended Herbicide Treatment Linked to Loss of About 25 Acres of Sesame in China

A 67-year-old farmer in Chuzhou, Anhui, reportedly used an AI-generated weed-and-pest-control plan to spray about 150 mu (25 acres) of sesame by drone. By the next day, the crop had largely withered. Agricultural technicians said the recommended mixture included fomesafen, a herbicide unsuitable for blanket application to sesame. The specific AI product and model have not been publicly confirmed.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The AI system failed to provide accurate and safe agricultural advice, recommending a lethal herbicide for sesame crops due to a lack of robust domain-specific knowledge.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: The farmer developed excessive trust in the AI over a year of use, leading him to apply dangerous chemical recommendations without verifying them with agricultural experts.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The crop damage was caused by incorrect chemical recommendations generated by the AI system after it was deployed to the public, which was an unintended outcome of its operation.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system is a general-purpose chatbot/AI assistant providing information. Under the EU AI Act, chatbots are classified as Limited Risk, requiring transparency obligations to ensure users are informed that they are interacting with an AI system.

AI system and alleged parties

• AI system: AI software
• AI purpose: Question Answering; Technical Text Generation
• Behaviour type: Assistant
• Alleged developer: Large language model developers, Chatbot developers
• Alleged deployer: Wu (Chuzhou farmer), Chatbot users
• Alleged harmed parties: Wu (Chuzhou farmer), Farmers in China, Farmers, Epistemic integrity, Chatbot users

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Minor, indirect Negligible
• Financial: direct Minor, indirect Negligible
• Environmental: direct Minor, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Minor
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Property

Reported: The report explicitly describes the destruction of 150 mu of sesame crops.

Directly caused: The AI's incorrect herbicide recommendation directly caused the complete withering and death of 150 mu (about 24.7 acres) of sesame seedlings.

Financial

Reported: The report explicitly describes an estimated financial loss of 150,000 RMB (approx $21,000 USD) for 1 occurrence.

Directly caused: The average financial loss per occurrence is approximately 150,000 RMB due to the complete destruction of the 150-mu sesame crop.

Environmental

Reported: The report explicitly describes the widespread spraying of an inappropriate herbicide across a 150-mu field.

Directly caused: The application of flusulfasulfaether, a broadleaf herbicide meant only for targeted use in soybean fields, across the entire 150 mu of sesame fields killed all vegetation including the crops and weeds.

Inferred additional harm: The widespread drone spraying of an incorrect chemical over 24.7 acres may have localized soil or ecological impacts, though specific long-term environmental damage is not detailed.

Psychological

Reported: The report explicitly describes emotional distress and heartache experienced by the farmer.

Indirectly caused: The farmer experienced significant distress and heartache ('心中满是心疼') upon seeing his entire 150 mu of sesame crops withered and destroyed overnight.

People affected

• Occurrences reported: 1
• People reportedly harmed: 1
• People reportedly exposed: 1

Potential causes

Management

• Inadequate Risk Control: AI developer failed to implement safety filters for chemical advice.
• Lack of Audience Adaptation: Failed to design interface warnings suitable for elderly users.

Technology

• No Independent Knowledge Base: AI relied on web-scraping without verifying domain-specific safety.
• Inappropriate Plan Recommendation: AI suggested full-field spraying instead of targeted application.
• Lack of Domain Safety Guardrails: System lacked safety filters for hazardous chemical recommendations.

Data Inputs

• Unverified Web Data Integration: AI synthesized crawled web data containing incorrect agricultural advice.
• Lack of Input Verification: No validation layer to check chemical compatibility before output.

Human Factors

• User Over-reliance on AI: Gradual trust over a year led the farmer to stop verifying suggestions.
• No Verification with Experts: The farmer did not consult agricultural technicians before spraying.
• Unnoticed Warning Label: The tiny warning text on the interface was missed by the elderly farmer.

Process and Methods

• Inconspicuous Warning Design: Warning was in small font at the top, easily missed by users.
• Lack of Domain-Specific Review: No expert review process for safety-critical agricultural recommendations.

Regulatory Environment

• Weak Warning Prominence Standards: Lack of strict enforcement of prominent warnings for high-risk fields.

Information quality

• Classification confidence: High
• Reason for confidence: The reports from multiple sources are highly consistent regarding the facts of the incident, the specific chemicals recommended, the acreage destroyed, and the financial loss. The role of the AI is explicitly documented with chat logs.
• Ambiguities identified: The specific name of the AI application is not mentioned, only referred to as 'AI software' or 'AI app'.
• Alternative interpretations: None. The causal link between the AI's recommendation, the farmer's action, and the crop destruction is clear and verified by agricultural experts.

A localized incident where a Chinese farmer suffered crop loss after executing incorrect herbicide advice from a commercial AI chatbot. The event represents a consumer product safety issue and user overreliance on unverified AI outputs, carrying negligible national security implications.

• Overall national security impact: Negligible
• Scope: Single nation
• Primary target: No clear primary
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. This represents a localized consumer safety and product liability issue rather than an active national security crisis requiring immediate response.
• Autonomy: Human-controlled. The AI provided recommendations in an advisory capacity, and the human user retained full control, making the final decision to apply the herbicide.
• Novelty: Established threat. AI hallucinations, incorrect synthesis of web data, and user overreliance on chatbot advice are well-documented and common occurrences.

Impact by dimension

• Physical security: Negligible. The incident is a highly localized agricultural failure affecting a single private farm of 24.7 acres, presenting no threat to critical infrastructure or national physical security.
• Information security: Negligible. No intelligence compromise, classified data theft, or systematic information warfare operations are associated with this commercial AI chatbot failure.
• Sovereignty: Negligible. The failure of a consumer AI application providing incorrect farming advice does not impact government operations, state authority, or sovereign decision-making.
• Economic security: Negligible. The financial loss of approximately $21,000 USD is localized to a single individual and has no bearing on strategic industries or national economic stability.
• Societal stability: Negligible. While causing personal financial and emotional distress to the farmer, the incident does not threaten social cohesion, civil liberties, or public safety at scale.