Purportedly Taxpayer-Funded Deloitte Report for Australian Government Contains Alleged AI-Generated Citations and Fabricated Legal Quote

Australian academics reportedly identified fabricated references and a misattributed legal quote in a $439,000 Deloitte report for the Department of Employment and Workplace Relations on welfare compliance. Deleted citations allegedly referred to non-existent works, and a Federal Court judgment was misstated. Deloitte later acknowledged limited use of generative AI in preparing the report and issued a corrected version with a partial refund.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The AI system generated false and misleading information in the form of non-existent academic citations and a fabricated court quote, which were then included in a government advisory report.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Deloitte employees over-relied on the AI system's outputs, failing to verify the fabricated references before submitting the report to the government.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The fabrication of references was an unintentional hallucination produced by the GPT-4o model during its post-deployment use by Deloitte.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system used was a general-purpose generative AI large language model (Azure OpenAI GPT-4o) used to generate text content, which falls under transparency obligations for AI-generated content.

AI system and alleged parties

• AI system: GPT-4o 4o (OpenAI)
• AI purpose: Writing Assistant; Technical Text Generation
• Behaviour type: Assistant
• Alleged developer: Unknown large language model developers
• Alleged deployer: Deloitte
• Alleged harmed parties: Scholars erroneously cited, Lisa Burton Crawford, Janina Boughey, Government of Australia, General public of Australia, Epistemic integrity, Department of Employment and Workplace Relations (DEWR), Deloitte, Carolyn Adams, Academics

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Minor, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Minor
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: The report explicitly describes financial losses, noting that Deloitte agreed to a partial refund of the 439,000 AUD contract by returning the final instalment.

Directly caused: Deloitte suffered a direct financial loss by refunding the final contract instalment to DEWR, and DEWR initially paid for a substandard report.

Inferred additional harm: Deloitte may face additional indirect financial losses from reputational damage and potential loss of future government contracts.

Epistemic

Reported: The report explicitly describes epistemic harm through the fabrication of academic citations and a court judgment quote.

Directly caused: The AI generated at least half a dozen non-existent academic references and a completely fabricated quote from a Federal Court judgment in the Amato case.

Indirectly caused: The incident led to the submission of a flawed, inaccurate advisory report to the Commonwealth government, potentially misstating the common law of Australia.

People affected

• Occurrences reported: 1
• People reportedly harmed: 5
• People reportedly exposed: 10

Potential causes

Management

• Poor Quality Assurance: Deloitte management approved a substandard, unverified report for delivery.
• Initial Lack of Transparency: Management avoided disclosing AI use when errors were first exposed.

Technology

• Generative AI Hallucinations: GPT-4o model generated nonexistent academic references and court quotes.
• Unreliable AI Tool Chain: The Azure OpenAI tool chain produced plausible-sounding but fake data.

Data Inputs

• Unverified Training or Prompt Data: The AI model lacked grounding in verified legal and academic databases.

Human Factors

• Inadequate Human Review: Deloitte personnel failed to verify AI outputs before publication.
• Over-reliance on Generative AI: Consultants relied on AI to perform core analytical and drafting tasks.
• Carelessness by Report Authors: Authors showed unfamiliarity with key legal facts of the Amato case.

Process and Methods

• Flawed Verification Processes: No robust process existed to cross-check cited academic and legal sources.
• Undisclosed AI Methodology: The use of generative AI was hidden in the initial version of the report.
• Flawed Error Correction Method: Deloitte initially replaced fake citations with other AI-generated fakes.

Regulatory Environment

• Weak Procurement Oversight: Government agencies lacked mechanisms to verify AI use by contractors.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, consistent, and detailed accounts of the incident, including the specific AI model used (Azure OpenAI GPT-4o), the nature of the errors (hallucinated citations and court quotes), the financial consequences (partial refund of a $439,000 contract), and statements from both Deloitte and the affected academics.
• Ambiguities identified: The exact amount of the refunded final instalment was not publicly disclosed.
• Alternative interpretations: None. The use of generative AI and the resulting hallucinations are explicitly admitted in the updated report methodology.

Deloitte Australia submitted a government report containing AI-generated fabricated academic citations and a false court quote. While representing a minor compromise to government advisory processes, the incident was quickly resolved through correction and a partial refund, posing negligible threat to national security.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: Australia
• Alleged perpetrator: Deloitte Australia

Threat characteristics

• Imminence: Long-term. The incident represents an ongoing strategic concern regarding the reliability of generative AI in government decision-making, rather than an immediate crisis.
• Autonomy: Human-controlled. The AI tool chain was used as a writing assistant, and humans retained final authority and responsibility for verifying and submitting the report.
• Novelty: Established threat. Generative AI hallucinations and the risk of overreliance on unverified LLM outputs are well-documented and established technical phenomena.

Impact by dimension

• Physical security: Negligible. The incident involved a consulting report and did not pose any threat to physical systems, critical infrastructure, or human safety.
• Information security: Negligible. No evidence of intentional disinformation campaigns, intelligence compromises, or foreign influence operations. The fabrications were unintentional AI hallucinations.
• Sovereignty: Minor. A government advisory report on welfare compliance contained fabricated citations and a false court quote, temporarily introducing misinformation into government policy processes before being corrected.
• Economic security: Negligible. The economic impact was limited to a partial refund of a 439,000 AUD contract, with no threat to strategic industries, financial systems, or technological competitive advantage.
• Societal stability: Negligible. While the report concerned welfare compliance, the errors were detected and corrected, resulting in no direct societal instability, mass surveillance, or systemic civil rights violations.