Canada Revenue Agency (CRA) AI Chatbot 'Charlie' Reportedly Gave Incorrect Tax Filing Guidance at Scale

Charlie the Chatbot, an AI-powered system deployed by the Canada Revenue Agency (CRA), has reportedly been providing inaccurate or incomplete tax-related information to members of the public. An audit by the Auditor General of Canada reportedly found the chatbot produced correct responses in fewer than half of tested cases. The system has been publicly available across multiple CRA webpages since March 2020 and reportedly used by millions of users.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The chatbot failed to perform reliably, providing incorrect answers to tax questions in a significant percentage of test cases and real-world interactions.

Additional risk subdomains

• 3.1 False or misleading information: The chatbot generated and spread incorrect tax information to users seeking assistance with their tax filings.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by the Charlie chatbot delivering incorrect answers to users after it was deployed to the public.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system is a chatbot, which under the EU AI Act falls under Limited Risk, requiring transparency obligations so users know they are interacting with an AI.

AI system and alleged parties

• AI system: Charlie the Chatbot
• AI purpose: Question Answering; Chatbot
• Behaviour type: Assistant
• Alleged developer: Unknown generative AI developers, Government of Canada, Canada Revenue Agency (CRA)
• Alleged deployer: Government of Canada, Canada Revenue Agency (CRA)
• Alleged harmed parties: General public of Canada, General public, Epistemic integrity, Canadian taxpayers

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Substantial, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: The report describes 1 ongoing incident of government waste causing a total financial loss of over $18 million in taxpayer funds spent on developing and operating the ineffective chatbot.

Directly caused: The CRA spent $18 million on the development and operation of the chatbot, which represents a direct loss of public funds on an ineffective tool.

Inferred additional harm: Taxpayers who received incorrect tax advice from the chatbot may have faced individual financial losses, penalties, or interest due to incorrect filings, though these losses are not quantified in the report.

Epistemic

Reported: The report explicitly describes epistemic harm in the form of the chatbot providing incorrect answers to tax-related questions.

Directly caused: The chatbot provided incorrect answers to 4 out of 6 test questions in the Auditor General's audit, and had an error rate of up to 30 percent in real-world use.

Inferred additional harm: With over 18 million questions asked, it is highly likely that many Canadian taxpayers were misled about their tax obligations, eroding trust in official government information.

People affected

• Occurrences reported: 1
• People reportedly exposed: 7000000

Potential causes

Management

• High Expenditure with Low Return: Spent 18 million dollars on a tool that delivered incorrect information.
• Inadequate Pre-Rollout Testing: Broadly rolled out the chatbot in 2021 before resolving accuracy issues.

Technology

• Inaccurate Chatbot Responses: The chatbot answered only 2 out of 6 basic tax questions correctly.
• Inferior AI Model Technology: Specialized government tool performed worse than free public AI tools.

Data Inputs

• Lack of Interaction Data Review: CRA could not verify real-world accuracy without manual transcript reviews.

Human Factors

• Premature Public Deployment: Deployed chatbot to the public while acknowledging it was still learning.

Process and Methods

• Low Internal Accuracy Standards: Internal testing accepted a low 70 percent accuracy threshold.

Information quality

• Classification confidence: High
• Reason for confidence: The reports are based on an official audit by the Auditor General of Canada, providing precise figures on cost ($18 million), usage (7 million conversations), and accuracy rates (33% to 70%). There is little ambiguity about the chatbot's performance and the public scrutiny it faced.
• Ambiguities identified: The exact number of taxpayers who made filing errors due to the chatbot's advice is not specified.

The Canada Revenue Agency's 'Charlie' chatbot provided inaccurate tax information to citizens, resulting in 18 million dollars in wasted public funds and potential taxpayer confusion. While illustrating significant challenges in government AI deployment and public trust, the incident carries negligible national security implications.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: Canada
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing public service delivery and government IT procurement issue rather than an immediate national security crisis.
• Autonomy: Human-controlled. The chatbot functions as an informational assistant, leaving final tax filing decisions and actions entirely to the human users.
• Novelty: Established threat. Large language models and rule-based chatbots providing incorrect or hallucinated information is a well-documented and established AI limitation.

Impact by dimension

• Physical security: Negligible. The incident involves a public-facing tax chatbot and poses no threat to physical systems, critical infrastructure, or human safety.
• Information security: Negligible. While the chatbot provided inaccurate tax information, this was an unintentional system limitation rather than a hostile disinformation campaign or intelligence compromise.
• Sovereignty: Minor. The failure of a government-deployed tool impacts service delivery and taxpayer trust, but it does not disrupt core constitutional processes or state authority.
• Economic security: Negligible. Although 18 million dollars in public funds was spent on an ineffective tool, this does not threaten macro-financial stability, critical supply chains, or strategic technological advantages.
• Societal stability: Negligible. The chatbot's errors do not represent systemic discrimination, mass surveillance, or direct threats to civil liberties and social cohesion.