Swedish Contraceptive App, Natural Cycles, Allegedly Failed to Correctly Map Menstrual Cycle

Some women using the contraceptive app, Natural Cycles, reported unwanted pregnancies, revealing its algorithm's difficulties in mapping menstrual cycles.

Risk classification

• Primary risk domain: 7 AI system safety, failures, & limitations
• Primary risk subdomain: 7.3 Lack of capability or robustness

The algorithm lacked the robustness to handle typical human variations, such as irregular menstrual cycles, stress, or minor temperature tracking errors, leading to incorrect fertility predictions.

Additional risk subdomains

• 5.1 Overreliance and unsafe use: Users placed a high level of trust in the app, treating it with the same expectations as a contraceptive pill, which led to unsafe reliance in a critical health scenario.

Causal factors

• Entity: AI
• Intent: Unintentional
• Timing: Post-deployment

The unwanted pregnancies were caused by the failure of the AI algorithm to robustly and accurately predict fertility windows under real-world conditions post-deployment.

EU AI Act risk tier

• Risk tier: 2 High Risk

High Risk: The system is a certified medical device used in healthcare for contraception, which has significant implications for physical health, safety, and fundamental rights.

AI system and alleged parties

• AI system: Natural Cycles algorithm (Natural Cycles)
• AI purpose: Health AI Assistant; Behavioral Modeling
• Behaviour type: Assistant
• Alleged developer: Natural Cycles
• Alleged deployer: Natural Cycles
• Alleged harmed parties: Women, Natural Cycles users

Harm severity

Severity ratings are based on this scale.

Highest direct severity in any category: Minor. Severity is scored from Negligible to Catastrophic in each harm category, for harm the reports describe as caused directly or indirectly by the AI system.

• Physical: direct Minor, indirect Negligible
• Infrastructure: direct Negligible, indirect Negligible
• Property: direct Negligible, indirect Negligible
• Financial: direct Minor, indirect Minor
• Environmental: direct Negligible, indirect Negligible
• Malicious content: direct Negligible, indirect Negligible
• Differential treatment: direct Negligible, indirect Negligible
• Civil rights: direct Negligible, indirect Negligible
• Democracy: direct Negligible, indirect Negligible
• Privacy: direct Negligible, indirect Negligible
• Psychological: direct Minor, indirect Minor
• Epistemic: direct Negligible, indirect Negligible
• Child sexual exploitation and abuse: direct Negligible, indirect Negligible

Physical

Reported: The reports explicitly describe physical harm in the form of unwanted pregnancies and subsequent abortions.

Directly caused: At least 37 women in Sweden sought abortions at one hospital after experiencing unwanted pregnancies while using the app. Additionally, individual users reported undergoing abortions or carrying unplanned pregnancies to term.

Inferred additional harm: Given that the app had over 700,000 users globally, it is highly likely that hundreds of other women experienced unwanted pregnancies and underwent medical procedures due to app failures.

Financial

Reported: The reports describe financial losses related to subscription fees and medical costs.

Directly caused: Users paid annual subscription fees of approximately 50 to 60 GBP for a service that failed to prevent pregnancy, with some unable to secure refunds.

Indirectly caused: Users incurred indirect costs associated with medical procedures for abortions or the expenses of childbirth and raising an unplanned child.

Inferred additional harm: Thousands of users globally likely incurred subscription losses and associated medical or child-rearing expenses due to app failures.

Psychological

Reported: The reports explicitly describe psychological distress, trauma, and anxiety resulting from the unplanned pregnancies.

Directly caused: Users reported experiencing a 'pit of despair', severe anxiety, shame, and trauma following unplanned pregnancies and subsequent abortions, which in some cases contributed to relationship breakdowns.

Indirectly caused: Users also reported high levels of stress and anxiety associated with the strict daily temperature-tracking requirements, such as waking up in panic.

Inferred additional harm: It is highly likely that many other users globally experienced significant emotional distress, anxiety, and trauma due to unexpected contraceptive failures.

People affected

• Occurrences reported: 1
• People reportedly harmed: 40
• People reportedly exposed: 700000

Potential causes

Management

• Misleading Marketing: Promoted app as as effective as the pill without behavioral context.
• Growth Prioritized Over Safety: Used aggressive influencer marketing targeting young, unsuitable users.
• Inadequate Risk Communication: Downplayed warnings that the app is not for those avoiding pregnancy.

Technology

• Inaccurate BBT Premise: BBT raises marginally and up to 75 percent of women show no rise.
• Algorithmic Cycle Mapping Failure: Algorithm fails to accurately predict ovulation for irregular cycles.
• Unstable Predictions: Fertility predictions changed from green to red within the same day.

Data Inputs

• Erratic Temperature Readings: Sleep disturbances, stress, and hangovers cause erratic temperature data.
• Measurement Sensitivity: Slightest movement before measuring temperature skews the data inputs.
• Missing Measurement Data: Users forgetting to measure daily or waking up at inconsistent times.

Human Factors

• Unprotected Sex on Red Days: Users had unprotected sex on days the app designated as high-risk.
• Over-reliance on App: Users trusted the app like a pill, neglecting behavioral requirements.
• Unsuitable User Profiles: Users with irregular cycles or PCOS used the app despite high risks.

Process and Methods

• Flawed Clinical Studies: Studies used self-selected existing users rather than randomized trials.
• No Pre-Purchase Screening: No mandatory screening to block users who would be devastated by pregnancy.
• Stressful Tracking Process: Demanding daily tracking routine caused high stress and anxiety.

Regulatory Environment

• Premature Certification: EU and FDA approvals gave a medical device status to a behavioral method.
• Lack of Marketing Oversight: Regulators allowed aggressive marketing without clear safety warnings.
• Lack of Standardized Testing: Lack of medicalized control tests required for contraceptive certification.

Information quality

• Classification confidence: High
• Reason for confidence: The reports provide detailed, consistent accounts of the app's functionality, the specific regulatory investigation in Sweden, and multiple personal testimonies of the resulting harms. The clinical efficacy statistics and the limitations of the technology are well-documented.
• Ambiguities identified: The exact proportion of pregnancies caused by algorithmic error versus user non-compliance is slightly ambiguous, though statistics on both are provided.
• Alternative interpretations: The incident could be interpreted primarily as a human behavioral failure (user error) rather than an AI failure, but the algorithm's lack of robustness under typical real-world conditions supports the AI safety classification.

An algorithmic failure in the Natural Cycles contraceptive app led to unwanted pregnancies and subsequent regulatory scrutiny in Sweden. As a consumer health product issue, this incident carries negligible national security implications across all assessed categories.

• Overall national security impact: Negligible
• Scope: Multiple nations
• Primary target: No clear primary
• Other affected: Sweden, United Kingdom
• Alleged perpetrator: Unknown

Threat characteristics

• Imminence: Long-term. Represents an ongoing consumer product safety and regulatory concern rather than an active national security crisis.
• Autonomy: Human-controlled. The AI acts as an assistant predicting fertility windows, but relies entirely on human data input and human compliance to be effective.
• Novelty: Established threat. Algorithmic failures in consumer health and tracking applications are known issues with established precedents.

Impact by dimension

• Physical security: Negligible. No threat to critical infrastructure, kinetic systems, or national physical security. The incident involves a consumer health application.
• Information security: Negligible. No intelligence compromise, state-sponsored disinformation, or information warfare operations are associated with this incident.
• Sovereignty: Negligible. No threat to state authority, election integrity, or core government operations.
• Economic security: Negligible. No strategic technology theft, financial system attacks, or national economic security implications.
• Societal stability: Negligible. While causing personal distress and health issues to individual users, the incident does not threaten societal stability or human rights at a national scale.