Automated decision systems.

Plain English Breakdown

The bill's status as 'Ordered to inactive file' suggests it may not become law, but this summary reflects its content regardless of current legislative status.

Regulating Automated Decision Systems

AB-1018 regulates how developers of automated decision systems (ADS) must evaluate and document these systems before deployment, and requires deployers to provide information about the system's impacts and allow appeals starting in 2027.

What This Bill Does

• Defines an 'automated decision system' as a computational process using machine learning or artificial intelligence that issues simplified output like scores, classifications, or recommendations to assist or replace human discretionary decisions impacting individuals.
• Requires developers of ADS to conduct performance evaluations and impact assessments before deploying the system.
• Requires deployers of ADS to provide information about the system's impacts and allow people affected by its decisions to appeal those decisions starting in 2027.
• Exempts certain documents related to ADS from public records laws when requested by the Attorney General for investigation purposes.

Who It Names or Affects

• Developers of automated decision systems
• Deployers and users of automated decision systems
• People affected by decisions made or facilitated by these systems

Terms To Know

Automated Decision System (ADS)

A computational process using machine learning, statistical modeling, data analytics, or artificial intelligence that issues simplified output to assist or replace human discretionary decisionmaking and materially impacts natural persons.

Performance Evaluation

An assessment of how well an automated decision system works and meets its intended goals.

Limits and Unknowns

• The bill does not specify what happens if a developer or deployer fails to comply with the requirements.
• It is unclear how this regulation will be enforced beyond allowing public entities like the Attorney General to bring civil actions for noncompliance.

Official Summary Text

AB 1018, as amended, Bauer-Kahan.
Automated decision systems.
The California Fair Employment and Housing Act establishes the Civil Rights Department within the Business, Consumer Services, and Housing Agency and requires the department to, among other things, bring civil actions to enforce the act.
Existing law requires, on or before September 1, 2024, the Department of Technology to conduct, in coordination with other interagency bodies as it deems appropriate, a comprehensive inventory of all high-risk automated decision systems that have been proposed for use, development, or procurement by, or are being used, developed, or procured by, any state agency.
This bill would generally regulate the development and deployment of an automated decision system (ADS) used to make consequential decisions, as defined. The bill would define “automated decision system” to
mean a computational process derived from machine learning, statistical modeling, data analytics, or artificial intelligence that issues simplified output, including a score, classification, or recommendation, that is designed or used to assist or replace human discretionary decisionmaking and materially impacts natural persons.
This bill would require a developer of a covered ADS, as defined, to take certain actions, including conduct
performance evaluations
impact assessments
of the covered ADS and provide deployers to whom the developer transfers the covered ADS with certain information, including
a high-level summary of
the results of those
performance evaluations.
impact assessments.
This bill would, beginning January 1, 2027, require a deployer of a covered ADS to take certain actions, including provide certain disclosures to a subject of a consequential decision made or facilitated by the covered ADS and provide the subject with an opportunity to appeal the outcome of the consequential decision, as prescribed.
This bill would require a developer, deployer, or auditor to, within 30 days of receiving a request from the Attorney General, provide an unredacted copy of the
performance evaluation
impact assessment
prepared pursuant to the bill to the Attorney General and would exempt those records from the California Public Records Act.
This bill would authorize certain public entities, including the Attorney General, to bring a specified civil action for noncompliance.
Existing constitutional provisions require that a statute that limits the right of access to the meetings of public bodies or the writings of public officials and agencies be adopted with findings demonstrating the interest protected by the limitation and the need for protecting that interest.
This bill would make legislative findings to that effect.

Current Bill Text

Download Bill PDF