South African Home Affairs Revised White Paper Reportedly Included Fictitious References Believed to Be AI-Generated

South Africa's Department of Home Affairs reportedly appended apparent AI-generated fictitious references to its Cabinet-approved Revised White Paper on Citizenship, Immigration and Refugee Protection. The department said the references were not cited in the body, withdrew the standalone list, suspended two senior officials, and commissioned an independent review of policy documents produced since November 2022. It maintained that the paper's substantive policy positions were unaffected.

Risk classification

• Primary risk domain: 3 Misinformation
• Primary risk subdomain: 3.1 False or misleading information

The AI system generated fictitious citations and references (hallucinations) which were included in official government policy documents, spreading false information.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Government officials relied on ChatGPT to generate factual references for official policy documents without verifying the outputs.

Causal factors

• Entity: Human
• Intent: Unintentional
• Timing: Post-deployment

The incident was caused by government officials unintentionally publishing unverified, hallucinated AI outputs in official government policy documents after the AI system was publicly deployed.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system involved is ChatGPT, a general-purpose chatbot and text generator. According to the definitions, chatbots and systems generating AI content fall under Risk Level 3, requiring transparency obligations.

AI system and alleged parties

• AI system: ChatGPT (OpenAI)
• AI purpose: Writing Assistant; Technical Text Generation
• Behaviour type: Assistant
• Alleged developer: Large language model developers, Generative AI developers
• Alleged deployer: South African Department of Home Affairs, Government of South Africa, Government agencies
• Alleged harmed parties: General public of South Africa, General public, Epistemic integrity, Citizenship, immigration, and refugee policy stakeholders

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Minor, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Epistemic

Reported: Yes, the report explicitly describes the generation of fictitious references and citations (hallucinations) in official policy documents.

Directly caused: The AI system generated fictitious references that were appended to the revised white paper on citizenship, immigration, and refugee protection, as well as the draft National AI Policy.

People affected

• Occurrences reported: 2
• People reportedly harmed: 2
• People reportedly exposed: 2

Potential causes

Management

• Inadequate Oversight of Drafting: Management failed to supervise the drafting and referencing process.

Technology

• LLM Hallucinations: Large language models generated fictitious and erroneous citations.
• Lack of Automated Verification Tools: No automated tools were used to verify the validity of the references.

Data Inputs

• Unverified Reference Lists: Citations were generated and attached after the fact without verification.
• Lack of Credible Data Sources: The LLM lacked access to or did not use credible data to generate sources.

Human Factors

• Overreliance on AI Output: Officials trusted AI-generated citations without performing manual checks.
• Lack of AI Literacy: Staff lacked understanding of LLM limitations like hallucination risks.

Process and Methods

• Absence of AI Review Processes: Internal approval processes lacked specific AI checks and declarations.
• Flawed Reference Compilation: References were attached after drafting without cross-referencing the text.

Regulatory Environment

• Lack of Government AI Guidelines: No clear public sector policies governed the safe use of AI in drafting.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, consistent, and explicit details about the AI system used (ChatGPT), the nature of the failure (hallucinated references), the consequences (suspensions, policy withdrawals, independent reviews), and the affected government departments. There are no major conflicting accounts or significant ambiguities regarding the core events.
• Ambiguities identified: The exact prompts used by the officials and the specific number of fictitious citations generated are not detailed.

South African government officials unintentionally included fictitious, AI-generated citations in official draft policy documents, including a white paper on immigration and the draft National AI Policy. This led to the suspension of officials and policy withdrawals, highlighting the risks of unverified AI use in state decision-making and policy formulation.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: South Africa
• Alleged perpetrator: South African government officials

Threat characteristics

• Imminence: Long-term. Represents an ongoing strategic and administrative concern regarding the secure integration of AI tools in public administration.
• Autonomy: Human-controlled. AI was used as a writing assistant tool by human officials who maintained final control over the documents, despite failing to verify the outputs.
• Novelty: Evolved capability. Represents an evolution of established LLM hallucination risks into formal government policy-making processes.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or human safety threats were involved in this incident.
• Information security: Negligible. No intelligence compromise, classified data theft, or active foreign disinformation campaigns occurred.
• Sovereignty: Minor. Unintentional inclusion of fictitious AI-generated references compromised the integrity of draft national policy documents, leading to official suspensions and policy withdrawals, but managed via standard administrative reviews.
• Economic security: Negligible. No significant economic warfare, strategic technology theft, or financial system attacks occurred, though minor administrative and legal costs were incurred.
• Societal stability: Negligible. No threats to social cohesion, mass surveillance, or systemic human rights violations resulted from the draft policy errors.