Ever AI Reportedly Deceived Customers about FRT Use in App

Ever AI, now Paravision AI, allegedly failed to inform customers about the development and use of facial recognition that facilitates the sale of customers’ data to various businesses, a business model that critics said was an egregious violation of privacy.

Risk classification

• Primary risk domain: 2 Privacy & Security
• Primary risk subdomain: 2.1 Compromise of privacy by obtaining, leaking or correctly inferring sensitive information

The company extracted biometric face embeddings and trained facial recognition models using private, non-public photos uploaded by users to a cloud storage app without their consent.

Additional risk subdomains

• 4.1 Disinformation, surveillance, and influence at scale: The harvested data was used to develop and train facial recognition models marketed to law enforcement and military for surveillance purposes.

Causal factors

• Entity: Human
• Intent: Intentional
• Timing: Post-deployment

The privacy violations and unauthorized data harvesting were caused by intentional human decisions to train facial recognition models using deployed consumer app data.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The report describes facial recognition technology intended for law enforcement and biometric identification, which falls under High Risk applications. The FTC complaint specifically notes the creation of face embeddings and biometric data from photos.

AI system and alleged parties

• AI system: Ever AI (Everalbum)
• AI purpose: Face Recognition; Data Grouping
• Behaviour type: Tool
• Alleged developer: Ever AI
• Alleged deployer: Ever AI
• Alleged harmed parties: Ever AI users

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Severe. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Negligible, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Negligible, indirect Negligible
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Severe, indirect Substantial
• Democracy: direct Negligible, indirect Minor
• Privacy: direct Severe, indirect Severe
• Psychological: direct Minor, indirect Negligible
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Civil rights

Reported: The report explicitly describes concerns from civil rights advocates regarding the erosion of privacy and the development of surveillance technology without consent.

Directly caused: The unauthorized extraction of biometric data from millions of individuals to build facial recognition models violates basic privacy rights.

Indirectly caused: The development of facial recognition technology marketed to law enforcement and military poses risks of mass surveillance and civil rights violations.

Inferred additional harm: The deployment of the resulting facial recognition algorithms by enterprise customers could lead to further undetected civil rights violations, such as wrongful identification or unauthorized tracking.

Democracy

Reported: The report explicitly mentions that facial recognition technology is slowly normalizing surveillance and eroding privacy, which can impact democratic norms.

Indirectly caused: The normalization of commercial and government surveillance through facial recognition technology threatens public anonymity and the freedom of assembly.

Privacy

Reported: The report explicitly describes massive privacy violations involving the unauthorized use of private photos to train facial recognition models and the retention of deactivated users' data.

Directly caused: Everalbum harvested biometric data (face embeddings) from billions of private photos uploaded by tens of millions of users without their informed consent.

Indirectly caused: Everalbum failed to delete the photos and videos of users who deactivated their accounts, retaining them indefinitely in violation of their terms.

Inferred additional harm: It is highly likely that millions of other users' private biometric data was permanently compromised and integrated into commercial AI models before the FTC ordered their deletion.

Psychological

Reported: The report explicitly describes users feeling shocked, uncomfortable, and experiencing an invasion of privacy upon learning their photos were used.

Directly caused: Users expressed feeling 'shocked', 'uncomfortable', and that the practice was 'creepy' and 'invasive' when they discovered their family photos were used to train surveillance tech.

Inferred additional harm: It is likely that many of the tens of millions of users experienced similar distress, anxiety, or feelings of violation of privacy upon learning about the unauthorized data harvesting, though these are not individually quantified.

People affected

• Occurrences reported: 1
• People reportedly harmed: 10000000
• People reportedly exposed: 10000000

Potential causes

Management

• Prioritizing Profit Over Privacy: Pivoted to B2B AI business model to achieve venture-scale growth.
• Deceptive Consumer Representation: Misrepresented data practices to maintain consumer trust and downloads.
• Lack of Ethical AI Oversight: No AI ethics advisor or principles existed during initial development.

Technology

• Default-on Facial Recognition: Facial recognition was automatically enabled for most users.
• Undisclosed AI Model Training: Billions of private photos were used to train facial recognition algorithms.
• Data Deletion Failure: System failed to delete photos from deactivated accounts, retaining them.

Data Inputs

• Non-Consensual Personal Photos: Private family photos were harvested for AI training without consent.
• Unconsented Biometric Extraction: Biometric face embeddings were derived from users' uploaded images.
• Combined Datasets: Private images were mixed with public datasets to train enterprise models.

Human Factors

• User Lack of Awareness: Users were unaware their private photos trained surveillance technology.
• Deceptive Privacy Explanations: Users were misled by warm branding and unclear privacy policy terms.
• Overlooking User Consent: Company assumed passive consent through unread terms of service.

Process and Methods

• Inadequate Disclosure Processes: AI training practices were buried in a long privacy policy.
• Flawed Deactivation Workflows: Processes failed to delete user data after account deactivation.
• Geographically Inconsistent Controls: Opt-out controls were restricted to specific regions with strict laws.

Regulatory Environment

• Absence of Federal AI Laws: Lack of federal facial recognition laws enabled invasive practices.
• Weak Initial Enforcement: Regulatory bodies historically allowed companies to keep ill-gotten models.
• Lack of Biometric Regulations: Inadequate legal frameworks to protect biometric data nationally.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide consistent, detailed accounts of Everalbum's actions, the FTC investigation, and the subsequent settlement. The facts regarding the unauthorized use of photos, the scale of the dataset, and the regulatory outcomes are well-documented across multiple reputable sources.
• Ambiguities identified: There is some ambiguity regarding whether any law enforcement or military agencies actually tested or used the technology, though the reports state no formal contracts were signed with them.

Everalbum covertly used billions of user photos to train facial recognition models, which it then marketed to law enforcement and military agencies. The FTC intervened, ordering the deletion of the illegally trained models. This incident highlights substantial risks to societal stability and privacy rights, demonstrating how consumer data can be weaponized for mass surveillance without user consent.

• Overall national security impact: Substantial
• Scope: Multiple nations
• Primary target: United States
• Other affected: Multiple nations
• Alleged perpetrator: Everalbum, Inc.

Threat characteristics

• Imminence: Long-term. The incident represents a long-term strategic concern regarding commercial surveillance and biometric data privacy rather than an active, immediate crisis.
• Autonomy: Human-controlled. The facial recognition system functions as a tool to match and classify images, operating under direct human control without autonomous agency.
• Novelty: Evolved capability. This represents an evolved capability where a standard consumer app leveraged massive user datasets to transition into a commercial surveillance developer.

Impact by dimension

• Physical security: Negligible. No physical threats, kinetic attacks, or critical infrastructure compromise were reported in connection with this incident.
• Information security: Minor. The unauthorized creation of a massive biometric database from global users poses minor intelligence security risks, though no active military deployment occurred.
• Sovereignty: Negligible. No disruption to state authority, elections, or core government decision-making processes was indicated.
• Economic security: Minor. The regulatory action forced the deletion of the company's core AI models, impacting its technological assets and commercial viability.
• Societal stability: Substantial. Massive unauthorized biometric harvesting of tens of millions of global users represents a substantial violation of privacy and human rights, leading to FTC intervention.