Chinese Tech Firms Allegedly Developed Facial Recognition to Identify People by Race, Targeting Uyghur Muslims

Various Chinese firms were revealed by patent applications to have developed facial recognition capable of detecting people by race, which critics feared would enable persecution and discrimination of Uyghur Muslims.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.1 Unfair discrimination and misrepresentation

The AI systems were specifically designed to profile and identify individuals based on their Uyghur ethnicity, directly enabling state-sponsored discrimination and surveillance.

Additional risk subdomains

• 4.1 Disinformation, surveillance, and influence at scale: The technology was designed to facilitate large-scale government surveillance and tracking of the Uyghur population.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The development and testing of ethnicity-detection AI systems were intentional human decisions aimed at profiling specific ethnic groups post-deployment.

EU AI Act risk tier

• Risk tier: 1 Unacceptable

Unacceptable Risk: The reports describe AI systems used for biometric categorization to profile and target individuals based on race and ethnicity, as well as state-sponsored mass surveillance, which are prohibited outright under the Act due to severe violations of fundamental rights.

AI system and alleged parties

• AI system: Alibaba Cloud facial recognition algorithm, Face++, Huawei facial recognition technology, SenseTime facial recognition system (Alibaba, Huawei, Megvii, SenseTime)
• AI purpose: Face Recognition; Identification
• Behaviour type: Tool
• Alleged developer: SenseTime, Megvii, Huawei, Baibu, Alibaba
• Alleged harmed parties: Uyghur people

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Substantial, indirect Severe
• Civil rights: direct Substantial, indirect Severe
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Substantial, indirect Severe
• Psychological: direct Negligible, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Differential treatment

Reported: Yes, the reports describe technologies designed to profile and identify individuals specifically based on their Uyghur ethnicity.

Directly caused: Alibaba Cloud developed a facial recognition algorithm that can identify a person's ethnicity or whether they are Uyghur, and Huawei patented techniques to identify pedestrians by race.

Indirectly caused: The systems enable the Chinese government to automate ethnic profiling and target specific minority groups for tracking and surveillance.

Inferred additional harm: The widespread deployment of these systems likely subjects millions of Uyghurs to systematic, automated discrimination and unequal treatment across China.

Civil rights

Reported: Yes, the reports explicitly mention systematic human rights violations being turbocharged by technology.

Directly caused: The AI systems were designed to alert the government when detecting Uyghur faces, directly violating their rights to privacy, non-discrimination, and freedom of movement.

Indirectly caused: The technology supports government projects mandating Uyghur analytics, facilitating the broader persecution and detention of Muslim minorities.

Inferred additional harm: The automated tracking of ethnic minorities likely leads to widespread, severe violations of fundamental civil liberties and human rights for millions of individuals.

Privacy

Reported: Yes, the reports mention a security breach of a smart city surveillance system hosted on Alibaba Cloud that could detect ethnicity.

Directly caused: The security breach exposed sensitive data from a surveillance system that detected and labeled individuals as Uyghur Muslims.

Indirectly caused: The development of facial recognition systems that automatically infer and log individuals' ethnicity violates their expectation of privacy.

Inferred additional harm: Millions of individuals are likely subjected to unauthorized, persistent surveillance and privacy violations as their biometric data and ethnicity are tracked without consent.

People affected

• Occurrences reported: 1

Potential causes

Management

• Lack of Algorithmic Governance: Corporate oversight failed to detect and stop discriminatory AI development.
• Compliance with State Surveillance: Prioritizing government contracts and mandates over ethical AI policies.

Technology

• Ethnicity Detection Algorithms: AI models trained specifically to identify Uighur and other ethnic groups.
• Automated Alerting Systems: Uighur alarm systems designed to automatically flag individuals to police.

Data Inputs

• Discriminatory Ethnic Tagging: Training data and product offerings labeled with explicit ethnic tags.
• Unmonitored Customer Databases: Sensitive profiling data hosted in cloud environments without access controls.

Human Factors

• Lack of Ethical Design Awareness: Engineers and researchers developing profiling systems without ethical review.

Process and Methods

• Flawed Patent Review Processes: Patents with discriminatory tech were approved and filed without intervention.
• Inadequate Product Testing Controls: Trial technologies tested in cloud environments without safety safeguards.

Regulatory Environment

• Government Surveillance Mandates: State projects mandating ethnic analytics and surveillance across regions.
• Absence of Discriminatory AI Bans: Lack of strong legal frameworks prohibiting automated ethnic profiling.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide clear, consistent accounts from multiple tech companies admitting to the existence of these patents or trial technologies, backed by investigations from IPVM. There is high certainty regarding the intended functionality of the AI systems.
• Ambiguities identified: The exact extent of active deployment versus trial testing is slightly ambiguous, as Alibaba claims the technology was not deployed by any customer, while other sources mention active government projects mandating Uyghur analytics.
• Alternative interpretations: None. The evidence of ethnicity-detection software development is clear and acknowledged by the companies involved.

The development and deployment of ethnicity-detection AI by Chinese tech firms to target Uyghur Muslims represents a severe violation of human rights and a significant advancement in state-sponsored mass surveillance capabilities. This incident has prompted coordinated international responses, including sanctions and technology export controls, highlighting its impact on global technology security.

• Overall national security impact: Severe
• Scope: Single nation
• Primary target: China
• Alleged perpetrator: Chinese government

Threat characteristics

• Imminence: Long-term. This represents an ongoing, long-term strategic capability and human rights concern rather than an immediate, fast-moving crisis.
• Autonomy: Human-supervised. The AI systems function as automated tools to detect and flag individuals, while human authorities make the final decisions and take action.
• Novelty: Evolved capability. This represents an evolved capability, advancing standard facial recognition technology to specifically target and profile individuals by ethnicity.

Impact by dimension

• Physical security: Negligible. No kinetic attacks or physical infrastructure damage were reported in connection with this facial recognition technology.
• Information security: Substantial. The technology represents a highly sophisticated, state-sponsored mass surveillance capability designed to track and profile specific populations.
• Sovereignty: Minor. While the system strengthens the Chinese state's domestic control, it does not directly threaten or compromise foreign sovereignty or democratic processes.
• Economic security: Minor. The development of these capabilities highlights China's advanced AI surveillance industry, leading to international sanctions and export controls on the involved firms.
• Societal stability: Severe. The AI systems directly enable state-sponsored systematic discrimination and mass surveillance of Uyghur Muslims, severely violating fundamental human rights on a population scale.