ChatGPT Implicated in Samsung Data Leak of Source Code and Meeting Notes

Samsung engineers are reported to have inadvertently leaked sensitive company data sometime in March 2023, including source code and internal meeting notes, by using ChatGPT to assist with tasks. The AI retained the inputted data, leading to a breach of confidentiality.

Risk classification

• Primary risk domain: 2 Privacy & Security
• Primary risk subdomain: 2.1 Compromise of privacy by obtaining, leaking or correctly inferring sensitive information

The incident involved the unauthorized transfer and potential exposure of Samsung's proprietary source code and internal meeting notes to OpenAI's servers through ChatGPT.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Employees overrelied on ChatGPT to perform work tasks involving highly sensitive proprietary data without considering the system's data retention policies.

Causal factors

• Entity: Human
• Intent: Unintentional
• Timing: Post-deployment

The data leak was caused by human employees unintentionally inputting confidential trade secrets into ChatGPT after its deployment.

EU AI Act risk tier

• Risk tier: 3 Limited Risk

Limited Risk: The system involved is ChatGPT, which is explicitly classified as a chatbot. Under the EU AI Act, chatbots are subject to transparency obligations to ensure users are aware they are interacting with AI.

AI system and alleged parties

• AI system: ChatGPT (OpenAI)
• AI purpose: Writing Assistant; Code Generation
• Behaviour type: Assistant
• Alleged developer: OpenAI
• Alleged deployer: Samsung engineers
• Alleged harmed parties: Samsung

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Negligible. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

People affected

• Occurrences reported: 3
• People reportedly exposed: 3

Potential causes

Management

• Delayed Risk Assessment: Management failed to assess risks of generative AI before adoption.
• Lack of Proactive AI Policies: Samsung reacted to leaks with warnings instead of proactive controls.

Technology

• AI Data Retention Policy: ChatGPT retains user input data to further train its models.
• Irretrievable Data Storage: Data sent to OpenAI servers cannot be retrieved or deleted.
• Lack of In-House AI Tools: No secure in-house AI alternative was available for employees.

Data Inputs

• Confidential Source Code Input: Engineers inputted proprietary source code to fix program bugs.
• Sensitive Meeting Notes Upload: Employee uploaded confidential meeting notes to generate a presentation.
• Proprietary Test Sequences Shared: Employee shared confidential chip fault optimization sequences with ChatGPT.

Human Factors

• Lack of Employee Awareness: Workers unwittingly leaked top secret data without realizing risks.
• Desire for Task Optimization: Employees prioritized work efficiency over data security protocols.
• Overreliance on AI Assistants: Workers trusted external AI tools with sensitive proprietary tasks.

Process and Methods

• Absence of AI Usage Guidelines: No clear processes existed for using external AI tools safely.
• Lack of Input Monitoring: No mechanism was in place to block confidential data uploads to AI.

Regulatory Environment

• Inadequate AI Data Regulation: Lack of enforced compliance regarding data deletion rights in AI.
• GDPR Non-Compliance: AI service lacks features to respect core tenets like data deletion.

Information quality

• Classification confidence: High
• Reason for confidence: The report clearly outlines the nature of the incident, the specific AI system used (ChatGPT), the developer (OpenAI), the affected party (Samsung), and the exact mechanism of the data leak (data retention for training). There is minimal ambiguity regarding what transpired.
• Ambiguities identified: None significant, though the exact contents of the leaked meeting notes and source code are kept confidential.
• Alternative interpretations: None. The event is straightforwardly a corporate data leak via an external AI tool.

Samsung semiconductor engineers inadvertently leaked proprietary source code and chip-testing data to OpenAI servers via ChatGPT. Because semiconductors are a highly strategic technology for South Korea, this exposure represents a substantial threat to technological and economic security, highlighting new vectors for intellectual property exfiltration via generative AI.

• Overall national security impact: Substantial
• Scope: Single nation
• Primary target: South Korea
• Alleged perpetrator: Samsung employees (unintentional)

Threat characteristics

• Imminence: Long-term. The leak represents a long-term strategic and economic risk regarding intellectual property loss rather than an immediate national security crisis.
• Autonomy: Human-controlled. The AI system served as a passive assistant, and the data exposure was entirely initiated and controlled by human employees.
• Novelty: Evolved capability. While corporate data leaks are established threats, utilizing generative AI chatbots as a vector for intellectual property exfiltration represents an evolved capability.

Impact by dimension

• Physical security: Negligible. No physical systems, critical infrastructure, or human safety elements were impacted or threatened by this incident.
• Information security: Minor. Proprietary source code and chip testing data were transferred to a third-party server, representing a minor intellectual property exposure but not a state-sponsored intelligence compromise.
• Sovereignty: Negligible. The incident was entirely corporate and did not impact South Korean government operations, electoral systems, or state sovereignty.
• Economic security: Substantial. Semiconductors are a critical strategic technology for South Korea's national interest. The leak of proprietary source code and chip-testing sequences to foreign servers represents a notable threat to technological competitive advantage.
• Societal stability: Negligible. No impact on civil liberties, societal stability, privacy of citizens, or human rights was reported.