ChatGPT Banned by Italian Authority Due to OpenAI's Lack of Legal Basis for Data Collection and Age Verification

The Italian Data Protection Authority alleged OpenAI lacked a justifiable legal basis for personal data collection and processing which facilitate training of ChatGPT, and lacked age-verification mechanism preventing exposure of the chatbot's inappropriate answers to children, prompting its ban.

Risk classification

• Primary risk domain: 2 Privacy & Security
• Primary risk subdomain: 2.1 Compromise of privacy by obtaining, leaking or correctly inferring sensitive information

The incident primarily involves allegations of unauthorized mass collection of personal data for training and a data breach exposing user conversations and payment details.

Additional risk subdomains

• 1.2 Exposure to toxic content: The regulator noted that the lack of age verification exposed minors to answers unsuitable for their development.
• 7.3 Lack of capability or robustness: The system suffered a data breach exposing user data and was noted to generate inaccurate personal information.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The privacy risks and subsequent regulatory ban were caused by OpenAI s intentional decisions regarding data collection and deployment practices after the model was trained.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system is a chatbot, which falls under the Limited Risk category requiring specific transparency obligations.

AI system and alleged parties

• AI system: ChatGPT (OpenAI)
• AI purpose: Chatbot; Question Answering
• Behaviour type: Assistant
• Alleged developer: OpenAI
• Alleged deployer: OpenAI
• Alleged harmed parties: Italian minors, Italian children

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Substantial, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Substantial, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Civil rights

Reported: The reports describe potential violations of European data protection rights (GDPR) which protect personal privacy.

Directly caused: The Italian regulator ordered the ban over suspected breaches of EU privacy rules (GDPR), specifically the lack of a legal basis for mass data collection to train algorithms.

Inferred additional harm: Millions of European users likely had their data privacy rights violated through unauthorized data processing and training without clear consent or opt-out mechanisms.

Privacy

Reported: The reports explicitly describe privacy violations, including a data breach and unauthorized personal data processing for training.

Directly caused: The Italian regulator blocked OpenAI from processing Italian users data due to a lack of legal basis for mass data collection and a data breach that exposed user conversations and payment information.

Inferred additional harm: A large number of Italian users had their private conversations and payment details exposed in the data breach, and millions had their personal data scraped for training without consent.

People affected

• Occurrences reported: 1

Potential causes

Management

• Prioritizing Growth Over Privacy: Rapid service rollout occurred before establishing robust privacy safeguards.
• Inadequate Risk Assessment: Management failed to align the model's data collection with EU privacy laws.
• Delayed Privacy Features: User privacy controls like incognito mode were introduced only after scrutiny.

Technology

• Algorithmic Data Leakage: A security vulnerability exposed user conversations and payment information.
• Generation of False Info: The model generated factually inaccurate personal information about individuals.
• Lack of Age Verification Tool: The system failed to verify user ages, exposing minors to unsuitable content.

Data Inputs

• Massive Unconsented Data Collection: Personal data was harvested from the internet without explicit user consent.
• Inaccurate Training Data: Training on raw web data led to the generation of false personal details.
• Lack of Initial Opt-Out Tools: Users lacked tools to object to their data being used for model training.

Human Factors

• Lack of Parental Consent: Minors aged 13 to 18 accessed the service without parental supervision.
• User Privacy Overreliance: Users shared sensitive personal and payment details during conversations.

Process and Methods

• No Default Privacy Mode: Chat history and training were enabled by default without an incognito option.
• Inadequate Age Verification Flow: The signup process did not verify if users were under 13 or lacked consent.
• Complex Data Objection Process: Exercising the right to object required a highly detailed, complex form.

Regulatory Environment

• GDPR Non-Compliance: The system lacked a valid legal basis under GDPR for mass data processing.
• Increased Regulatory Scrutiny: European regulators launched investigations into generative AI privacy risks.
• Lack of EU Representative Presence: OpenAI lacked a proactive local EU presence to handle compliance issues.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent and detailed information regarding the regulatory actions, the specific privacy concerns, and the technical remedies implemented by OpenAI.
• Ambiguities identified: The exact number of users affected by the data breach or exposed to false information is not specified in the reports.
• Alternative interpretations: None. The incident is clearly a regulatory compliance and privacy issue.

The temporary ban of ChatGPT in Italy by the Garante highlights growing regulatory scrutiny over AI data privacy and GDPR compliance. The incident represents a minor national security concern focused on consumer privacy rights and regulatory governance rather than a threat to critical infrastructure or state sovereignty.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: Italy
• Alleged perpetrator: OpenAI

Threat characteristics

• Imminence: Long-term. Represents an ongoing regulatory and compliance concern rather than an active national security crisis.
• Autonomy: Human-controlled. The AI system operates as a conversational assistant generating text based on direct user prompts.
• Novelty: Evolved capability. Represents an evolved regulatory challenge concerning the application of existing data privacy laws to generative AI models.

Impact by dimension

• Physical security: Negligible. No physical systems, infrastructure, or human safety risks were impacted by this regulatory privacy incident.
• Information security: Negligible. The incident involved a commercial data breach and privacy dispute, with no evidence of state-sponsored intelligence compromise or information warfare.
• Sovereignty: Negligible. The incident represents standard regulatory enforcement of data privacy laws by a sovereign authority, rather than a threat to government functions.
• Economic security: Negligible. Temporary commercial suspension of ChatGPT in Italy had negligible impact on strategic industries or national economic security.
• Societal stability: Minor. Involved potential violations of personal data privacy rights under GDPR and minor exposure risks, but lacked threats to broader societal stability.