AI safety research

Safety is the mandate, not a chapter

The law that creates CalCompute leads with safety. Government Code § 11546.8(b) directs the consortium to develop a framework that "advances the development and deployment of artificial intelligence that is safe, ethical, equitable, and sustainable" — first by "fostering research and innovation that benefits the public." Senate Bill 53's author put it plainly: CalCompute exists to "conduct research into the safe and secure deployment of large-scale artificial intelligence (AI) models," and to "demonstrate that safety does not stifle success."

Safety is not one of the framework report's seven lettered elements. It is the purpose clause every element answers to. Our research makes that clause concrete — so the report the consortium delivers to the Legislature treats safety as infrastructure, not an afterthought.

The problem: safety research is locked out

The researchers whose job is to find flaws in AI systems cannot get near them. The compute gap is stark — and the people are following the compute.

Access is gated as tightly as hardware. The California Report on Frontier AI Policy documents that "companies disincentivize safety research by implicitly threatening to ban independent researchers that demonstrate safety flaws in their systems" — terms of service that turn finding a flaw into grounds for a ban. In March 2024, over 350 leading AI researchers and advocates signed an open letter calling for a safe harbor for independent AI evaluation.

"The public sector is now significantly lagging in resources and talent compared to that of industry. This will have profound consequences because industry is focused on developing technology that is profit-driven, whereas public-sector AI goals are focused on creating public goods."

— Dr. Fei-Fei Li, co-lead of the Joint California Policy Working Group on AI Frontier Models

Our approach: trust but verify, made concrete

California already has its safety blueprint. The California Report on Frontier AI Policy — commissioned by Governor Newsom and led by Jennifer Tour Chayes, Mariano-Florentino Cuéllar, and Fei-Fei Li, published June 17, 2025 — roots the state's approach in an ethos of "trust but verify": third-party evaluation with safe harbors for evaluators, whistleblower protections, public-facing information sharing, adverse event reporting, and adversarial testing. Its warning is direct: "Transparency and independent risk assessment are essential to align commercial incentives with public welfare."

SB 53's transparency provisions now require large frontier developers to publish safety frameworks, use third parties to assess catastrophic risk, and report critical safety incidents to the state. But verification is only as strong as the people doing the verifying. Our research answers the question the report leaves to CalCompute: what must a public compute platform provide so independent safety research can actually happen? We study three requirements:

• Compute access for independent evaluators. Third-party risk assessment, red-teaming, and adversarial testing need serious hardware — the kind universities and public-interest researchers currently cannot get.
• Evaluation infrastructure. A fully owned and hosted public platform, with the human expertise the statute requires, where safety evaluations run outside any developer's terms of service.
• A public evidence base. The report's principles call for policymaking grounded in empirical research. Public compute makes the evidence on AI risk public property, not a commercial asset.

Where this lands in the framework report

Safety research threads through the framework report's statutory elements rather than standing alone. It shapes use parameters (Element D — which safety projects earn scarce public compute), governance (Element C — who safeguards researcher independence), and the landscape analysis (Element A — the evaluation-infrastructure gap no commercial cloud fills). The consortium delivers the framework report to the Legislature on or before January 1, 2027. Every finding on this page traces to the primary sources, starting with the full text of SB 53.