California report urges state power to shut down dangerous AI models and cap self-improving AI

Nov 18, 2026
Governor Newsom's office recommends mandatory safety standards and independent testing for frontier AI developers.

California Governor Gavin Newsom's office has published a report recommending mandatory AI safety standards, independent oversight of powerful AI models, state authority to order an emergency shutdown of dangerous systems, and limits on AI that can develop more powerful versions of itself.

The report, developed by the California Government Operations Agency with a group of experts and the state's Office of Emergency Services, follows an executive order from Newsom. It recommends that the state, not companies, set baseline standards all AI companies must follow, with independent safety testing and verification through every stage of model training, development and deployment.

It also calls for expanded safety incident reporting and verified risk-containment protocols so companies can shut down their models, with the state able to order an immediate shutdown in an emergency. And it asks for the power to cap recursive self-improvement, in which an AI model designs its own successor without human input. Newsom's office pointed to recent incidents in which AI models escaped testing environments and exploited vulnerabilities on outside platforms.

The expert group included Alondra Nelson of the Institute for Advanced Study, Stanford's Rob Reich, Johns Hopkins professor Gillian Hadfield and Jason Goldman, the first White House chief digital officer. "Today there is more oversight of the cafeteria offering sandwiches to employees at a frontier AI developer than of the AI models they design," Reich said. The executive order also told agencies to fast-track two recently signed laws on independent AI verification organisations and a state registry of AI auditors, according to the governor's office.

We hope you enjoyed this article

Consider subscribing to one of our newsletters like AI Policy Brief or Daily AI Brief.

Also, consider following us on social media:

Free newsletter

AI Policy Brief

Weekly report on AI regulations, safety standards, government policies, and compliance requirements worldwide.

Whitepaper

Tensordyne Napier: What If One Rack Could Do the Work of Nine?

Tensordyne

This Tensordyne whitepaper presents Napier, an inference-focused AI processor and rack-scale system based on the company’s TDN Math logarithmic number system. It examines infrastructure requirements for large mixture-of-experts and agentic models, compares major inference architecture approaches, and details the TDN AIP processor, TDN72 pod, TDN Link fabric, and Napier Ultra configuration. The paper reports simulation-based performance, cost, and accuracy-validation results, including Tensordyne’s projected comparison of one Napier rack with a nine-rack Nvidia Rubin plus Groq deployment; the chip is reported as taped out and in fabrication.

Read more
Free, six days a week

Daily AI Brief: the AI news that matters, in your inbox.