Job Screening Service Halts Facial Analysis of Applicants

In January 2021, HireVue removed the controversial AI expression tracking tool from its virtual job interview software.

Risk classification

• Primary risk domain: 1 Discrimination & Toxicity
• Primary risk subdomain: 1.1 Unfair discrimination and misrepresentation

The primary concern raised by critics and the EPIC complaint is that the AI system's unproven scoring of facial expressions and speech patterns could unfairly discriminate against candidates based on race, gender, or neurological differences.

Additional risk subdomains

• 7.3 Lack of capability or robustness: The system was heavily criticized by scientists as 'digital snake oil' and 'pseudoscience' because current AI cannot reliably infer psychological traits or job performance from facial movements.
• 2.1 Compromise of privacy by obtaining, leaking or correctly inferring sensitive information: The system collected and analyzed sensitive biometric data (facial geometry and voice prints) without clear consent, transparency, or user understanding of how their data was used.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The risk of discriminatory hiring and privacy violations occurred as an unexpected outcome of deploying the AI system to automate candidate screening.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The report describes an AI system used in recruitment and employment, which is explicitly classified as High Risk under the EU AI Act.

AI system and alleged parties

• AI system: HireVue AI hiring system (HireVue)
• AI purpose: Application Evaluation; Automatic Skill Assessment
• Behaviour type: Assistant
• Alleged developer: HireVue
• Alleged deployer: HireVue
• Alleged harmed parties: job applicants using HireVue, HireVue customers

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Substantial. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Minor, indirect Minor
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Minor, indirect Negligible
• Civil rights: direct Minor, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Substantial, indirect Negligible
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Financial

Reported: Yes, the report describes candidates being denied job opportunities and expending resources to find alternative employment.

Directly caused: Candidates were rejected from jobs (such as investment banking and teller positions) based on automated rankings, resulting in lost employment opportunities.

Indirectly caused: Rejected candidates had to expend significant time and financial resources to identify alternative employment opportunities.

Inferred additional harm: With over a million candidates screened by an unproven algorithm, it is highly likely that thousands of qualified applicants suffered significant financial losses due to unfair rejection.

Differential treatment

Reported: Yes, the report describes concerns that the system unfairly penalizes non-native speakers, people with stutters, or those with neurological differences.

Directly caused: The AI analyzed facial movements and tone, which critics argue acts as a 'license to discriminate' against individuals who do not fit the standard model.

Inferred additional harm: It is highly likely that systemic differential treatment occurred across the millions of screened candidates, favoring those who matched the demographic and behavioral profiles of the training data.

Civil rights

Reported: Yes, the EPIC complaint alleges violations of privacy rights and fair employment opportunities.

Directly caused: The system collected and analyzed biometric data (facial movements, voice) without meaningful consent or transparency.

Inferred additional harm: Systemic violations of privacy and civil rights likely affected a large portion of the millions of job seekers screened by the opaque system.

Privacy

Reported: Yes, the EPIC complaint explicitly alleges that HireVue's intrusive collection and secret analysis of biometric data causes substantial privacy harms.

Directly caused: The system collected candidates' facial geometry and voice biometrics without clear consent or transparency.

Inferred additional harm: Millions of job applicants had their biometric data harvested and analyzed by proprietary, opaque algorithms without their informed consent.

Psychological

Reported: Yes, the report explicitly describes candidates experiencing severe anxiety, alienation, and feeling dehumanized.

Directly caused: Candidates like Vartuli and the 25-year-old in Queens experienced intense 'interview anxiety' and felt dehumanized trying to perform for an opaque algorithm.

Inferred additional harm: Given that over a million candidates were screened, it is highly likely that thousands of other applicants suffered similar psychological distress and anxiety.

People affected

• Occurrences reported: 1
• People reportedly harmed: 2
• People reportedly exposed: 1000000

Potential causes

Management

• Prioritizing Speed Over Validity: Focus on reducing hiring time and costs over ensuring scientific validity.
• Misleading Product Claims: Representing that the system does not use facial recognition technology.
• Lack of Risk Assessment: Failure to evaluate whether benefits outweigh privacy and bias risks.

Technology

• Flawed Facial Emotion Science: Facial movements do not reliably reveal psychological traits or emotions.
• Opaque Predictive Algorithms: Secret algorithms make it impossible to know how scores are determined.
• Inability to Verify Validity: System lacks proven scientific accuracy, reliability, or replicability.

Data Inputs

• Biometric Data Collection: Intrusive collection of facial movements, voice tone, and word choice.
• Biased Historical Training Data: Training on past employees risks perpetuating existing demographic biases.
• Vulnerability to Environmental Noise: Background clutter and noise can skew candidate assessment scores.

Human Factors

• Candidate Anxiety and Stress: Alienating and dehumanizing process causes severe interview anxiety.
• Overreliance on Algorithmic Scores: Recruiters blindly trust machine decisions over human evaluations.
• Attempts to Game the System: Candidates try to trick the AI by altering natural behavior and expressions.

Process and Methods

• Lack of Independent Auditing: Company declined to make the assessment system available for outside audits.
• No Candidate Feedback Loop: Candidates cannot view their scores or challenge the automated decisions.
• Proprietary Assessment Models: Secret models prevent external evaluation and scientific peer review.

Regulatory Environment

• Lagging AI Regulations: Lack of federal oversight allowed unchecked use of biometric assessments.
• State Transparency Mandates: New laws like the Illinois act force disclosure and candidate consent.
• FTC Deceptive Practice Complaints: EPIC filed complaints over misleading claims about facial recognition.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide detailed accounts of the FTC complaint, the functionality of HireVue's software, the scientific criticisms, and the experiences of affected job candidates. The facts are consistent across multiple reputable sources.
• Ambiguities identified: There is some ambiguity regarding the exact number of candidates who were unfairly rejected or experienced severe distress, as well as the precise inner workings of HireVue's proprietary scoring algorithm.
• Alternative interpretations: HireVue argues that the system reduces human bias and is more objective than traditional recruiting, presenting an alternative view where the system is beneficial rather than harmful.

A commercial AI-driven hiring tool faced regulatory and public backlash over potential algorithmic bias and biometric privacy concerns. The incident highlights domestic challenges in AI governance and civil rights but presents minor direct national security implications.

• Overall national security impact: Minor
• Scope: Single nation
• Primary target: United States
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing regulatory and societal concern regarding algorithmic bias rather than an active, immediate security crisis.
• Autonomy: Human-supervised. The AI system generated employability scores to assist human recruiters, who maintained ultimate hiring authority.
• Novelty: Evolved capability. Represents a significant advancement in the commercial deployment of biometric and behavioral AI analysis in the recruitment sector.

Impact by dimension

• Physical security: Negligible. No physical security threats, kinetic attacks, or critical infrastructure disruptions were reported.
• Information security: Negligible. No evidence of intelligence compromise, classified data theft, or systematic information warfare operations.
• Sovereignty: Negligible. The incident involves a commercial hiring tool used by private companies, with no direct threat to state authority or government operations.
• Economic security: Negligible. The incident involves commercial HR software and does not threaten strategic national industries, financial systems, or technological competitive advantage.
• Societal stability: Minor. Concerns over algorithmic discrimination and biometric privacy affected job seekers but were managed through standard regulatory and legal channels, presenting minor national security impact.