Accidental Exposure of 38TB of Data by Microsoft's AI Research Team

Microsoft's AI research team accidentally exposed 38TB of sensitive data while publishing open-source training material on GitHub. The exposure included secrets, private keys, passwords, and internal Microsoft Teams messages. The team utilized Azure's Shared Access Signature (SAS) tokens for sharing, which were misconfigured, leading to the wide exposure of data.

Risk classification

• Primary risk domain: 2 Privacy & Security
• Primary risk subdomain: 2.1 Compromise of privacy by obtaining, leaking or correctly inferring sensitive information

The incident directly resulted in the exposure of 38TB of private data, including personal computer backups, passwords, and internal communications of Microsoft employees.

Additional risk subdomains

• 2.2 AI system security vulnerabilities and attacks: The misconfigured token granted write permissions on model files using Python's pickle format, creating a supply chain vulnerability where malicious code could be injected.

Causal factors

• Entity: Human
• Intent: Unintentional
• Timing: Post-deployment

The leak was caused by a human error in configuring an Azure SAS token, which occurred unintentionally while sharing trained models publicly.

EU AI Act risk tier

• Risk tier: 4 Minimal or No Risk

Minimal or No Risk: The open-source AI models being shared were for image recognition, which generally poses low or negligible risk to users and society under the EU AI Act.

AI system and alleged parties

• AI system: Microsoft AI research team (Microsoft)
• AI purpose: Image Classification
• Behaviour type: unknown
• Alleged developer: Microsoft's AI Research Division
• Alleged deployer: Microsoft
• Alleged harmed parties: Third parties relying on the confidentiality of the exposed data, Microsoft employees, Microsoft

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Minor, indirect Negligible
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Privacy

Reported: Yes, the report explicitly describes a massive leak of private data including workstation backups, passwords, keys, and internal messages.

Directly caused: Microsoft accidentally exposed 38TB of private data, including workstation backups of two employees and over 30,000 internal Teams messages from 359 employees.

Inferred additional harm: It is possible that credentials or keys exposed in the leak could have been accessed by unauthorized third parties during the three years the token was active, potentially leading to further unauthorized access, though no active exploitation was confirmed.

People affected

• Occurrences reported: 1
• People reportedly harmed: 359
• People reportedly exposed: 359

Potential causes

Management

• Lack of Security Visibility: Security teams lacked visibility into AI research and development processes.
• No CSPM Policies Enforced: Management failed to enforce policies to disable SAS token usage.
• No Dedicated Public Storage: Management did not mandate separate accounts for public and private data.

Technology

• Flawed SAS Token Design: Tokens are created client-side and lack centralized tracking in Azure.
• Insecure Pickle Model Format: TensorFlow ckpt files use pickle, allowing arbitrary code execution.
• Difficult Token Revocation: Revocation requires rotating the account key, affecting all other tokens.

Data Inputs

• Co-mingled Storage Account: Public AI datasets were stored alongside private workstation backups.
• Massive Training Datasets: AI research requires handling massive datasets, increasing exposure risks.

Human Factors

• Overly Permissive Token Config: Developers configured full control instead of read-only access.
• Excessive Token Expiry Date: Developers configured the token to expire in the year 2051.
• Wide Token Scope Selection: Developers shared the entire storage account instead of specific files.

Process and Methods

• Inadequate Secret Scanning: Lack of automated scanning to detect leaked SAS tokens in GitHub.
• Lack of Storage Account Logging: Storage logging was not enabled, preventing detection of active token use.
• Poor External Sharing Process: Account SAS was used for external sharing instead of safer alternatives.

Information quality

• Classification confidence: High
• Reason for confidence: The report provides a highly detailed technical analysis of the misconfiguration, the specific Azure features involved, the exact volume of data exposed, and a clear timeline of discovery and mitigation. Microsoft's confirmation of the issue further solidifies the credibility of the information.
• Ambiguities identified: None. The technical details of the SAS token misconfiguration and the nature of the exposed data are clearly explained.
• Alternative interpretations: None. The event is clearly a cloud security misconfiguration leading to data exposure.

Microsoft's AI research team accidentally exposed 38TB of sensitive data, including backups and internal communications, due to a misconfigured Azure SAS token. While the immediate threat was mitigated by invalidating the token, the incident highlights critical security risks in managing large-scale AI data and potential model supply chain vulnerabilities.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. The token has been invalidated, meaning there is no immediate threat, but it highlights long-term systemic risks in AI data management.
• Autonomy: Human-controlled. The incident resulted entirely from human misconfiguration of a storage token, not autonomous AI actions.
• Novelty: Evolved capability. While cloud leaks are common, the exposure of massive AI training datasets and potential for model-based supply chain attacks represents an evolved risk vector.

Impact by dimension

• Physical security: Negligible. The incident was a data leak with no physical impact or threat to critical infrastructure.
• Information security: Minor. Exposure of internal communications and keys could potentially aid intelligence collection, but there is no evidence of active exploitation by foreign adversaries.
• Sovereignty: Negligible. No disruption to government decision-making, electoral systems, or constitutional processes.
• Economic security: Minor. Accidental exposure of internal workstation backups and keys poses a minor risk to the company's technological security and creates potential supply chain vulnerability.
• Societal stability: Negligible. No mass surveillance, systematic discrimination, or civil unrest resulted from this data exposure.