Legal Advisory on the Application of Existing California Law to Artificial Intelligence in Healthcare (California Attorney General)

• Authority: California
• Parent authority: State governments
• Jurisdiction: United States
• Collections: U.S. state and local documents
• Most recent activity: Enacted
• Most recent activity date: 2025-01-13

Risk subdomain coverage

Good coverage — Comprehensive governance measures or mitigations with clear procedures relating to the risk subdomain are explicitly described.

• Unfair discrimination and misrepresentation
• Unequal performance across groups
• Compromise of privacy
• False or misleading information
• Overreliance and unsafe use
• Loss of human agency and autonomy
• Lack of capability or robustness
• Lack of transparency or interpretability

Minimal coverage — Brief mention but no specific mitigations, recommendations or governance measures relating to the risk subdomain are described.

• Fraud, scams, and targeted manipulation
• Governance failure

All other risk subdomains had no coverage in this document.

Risk Coverage Summary

The document has good coverage of approximately 10-12 subdomains, with strong focus on discrimination and toxicity (1.1, 1.3), privacy compromise (2.1), false information (3.1), fraud and manipulation (4.3), overreliance (5.1), loss of agency (5.2), governance failure (6.5), and system safety failures (7.3, 7.4). Coverage is concentrated in discrimination, privacy, misinformation, and AI system reliability domains.

Sector coverage

Good coverage — Comprehensive governance measures or mitigations with clear procedures relating to the sector are explicitly described.

• Health Care & Social Assistance

Minimal coverage — Brief mention but no specific mitigations, recommendations or governance measures relating to the sector are described.

• Finance & Insurance

All other sectors had no coverage in this document.

Sectors Summary

This document exclusively governs the Health Care and Social Assistance sector, with comprehensive coverage of AI use by healthcare providers, insurers, hospitals, and related healthcare entities in California. It does not govern other sectors.

Legislative Status

Soft Law

Legislative Status Summary

This is a legal advisory document that provides guidance on the application of existing California laws to AI in healthcare. It uses advisory language ('should ensure', 'should consider') and does not create new binding obligations, but rather interprets how existing hard law applies to AI systems.

Actors

Proposer

• Entity Types: AI_Governance_Actor
• Entities: California Attorney General's Office (AGO)
• Summary: The document explicitly states it is issued by the California Attorney General's Office to provide guidance on AI use in healthcare.

Target

• Entity Types: AI_Developer, AI_Deployer
• Entities: healthcare providers; insurers; vendors; investors; healthcare entities; hospitals; medical professionals; physicians; health care service plans; electronic health records (EHR) companies; digital health companies
• Summary: The advisory explicitly targets healthcare providers, insurers, vendors, and other healthcare entities that develop, sell, and use AI systems. It addresses both developers of AI systems and deployers (healthcare providers and insurers using AI).

Enforcer

• Entity Types: AI_Governance_Actor
• Entities: California Attorney General's Office; California Department of Managed Health Care; California Department of Insurance; FDA (Food and Drug Administration); relevant state agencies
• Summary: The document references enforcement through existing California laws and regulatory agencies. The AGO itself enforces the Unfair Competition Law and other consumer protection statutes. Other state agencies are mentioned for inspection and audit authority.

Monitor

• Entity Types: AI_Governance_Actor
• Entities: California Attorney General's Office; relevant state agencies; California Department of Managed Health Care; California Department of Insurance
• Summary: The document indicates that state agencies have authority to inspect and audit AI systems used by health care service plans, and the AGO is actively investigating AI use in healthcare.

AI Lifecycle Stages

Lifecycle stages covered by the documents, as defined in the OECD Framework for the Classification of AI Systems

• Plan and Design
• Collect and Process Data
• Build and Use Model
• Verify and Validate
• Deploy
• Operate and Monitor

AI Lifecycle Summary

The document addresses multiple AI lifecycle stages with primary focus on deployment and operation/monitoring. It covers design considerations (non-discrimination requirements), data collection (privacy protections), model development (testing and validation requirements), and strongly emphasizes deployment requirements and ongoing monitoring obligations.

Technical Scope

Terms relating to technical scope that are mentioned or implicitly referenced in the document

• AI Models
• AI Systems
• Generative AI

Technical Scope Summary

The document broadly refers to 'artificial intelligence (AI) and other automated decision systems' without defining specific technical categories. It mentions AI systems, AI models, and automated decisionmaking tools but does not explicitly distinguish between frontier AI, general purpose AI, foundation models, or other technical classifications. It does reference generative AI specifically in examples.