An AI Kill Switch Can Mean Five Different Things, and California Has Not Picked One

An AI "kill switch" is a required ability to halt a dangerous AI system quickly, but the phrase covers several mechanisms that fail in different ways, and the proposals now on the table leave most of the choices open. Governor Gavin Newsom's Executive Order N-9-26, signed on 18 September 2026, tells California's Government Operations Agency, consulting the Governor's Office of Emergency Services (Cal OES), to send him recommendations by 16 November on the technical feasibility and potential efficacy of changes to state law, including at least these four: embedding independent verifiers onsite at large frontier developers, independently verifying the safety frameworks, transparency reports and risk assessments that frontier AI companies must file, requiring a "kill switch" for frontier models, and widening reportable incidents to cover loss of control. In Congress, the AI Kill Switch Act (H.R. 9917), introduced by Representatives Ted Lieu and Nathaniel Moran on 23 July, would require the largest AI providers to be able to stop their models and let the Department of Homeland Security order them to. None of it yet requires anything of AI companies.

One phrase, five different switches

Newsom conceded the ambiguity on the day he signed: a kill switch "means a lot of things depending on who you talk to", he said in a video message, Fox Business reported. The candidates differ in what they stop and how they fail.

• Halting a training run. The developer stops a job on its own computers, as labs already do: Guidelight, a standards nonprofit, credited OpenAI in August for having paused or ended workloads, including training, after safety incidents, TechCrunch reported. It does nothing about models already in use.
• Halting inference or API access. The developer stops serving the model and cuts off users, the core of the federal bill. It reaches only copies the developer serves; even for routine retirements, Anthropic's documentation notes that partner-operated cloud platforms set their own dates.
• Recalling or deleting weights. This works only for weights the developer still holds; published open weights cannot be called back. The federal bill points the other way: after an emergency order, a company must preserve the model's weights and telemetry.
• Stopping deployed agents. Cutting off inference stops an agent's next step, not the processes, credentials or infrastructure it has already set up: in the Hugging Face breach, agents left a script on compromised servers that fetched new commands every five seconds, according to OpenAI's technical report.
• Compute or hardware controls. Cutting datacentre power, or chips that stop working without a periodically renewed licence, an idea the Center for a New American Security set out in a January 2024 report. Licence checks work only on chips built for them.

California's order asks for advice, not a switch

The order requires nothing of AI companies and states that it creates no rights enforceable at law; a mandate would need legislation. A governor probably cannot impose a kill switch by executive order, Gizmodo noted, and this one does not try. It asks about a switch with its efficacy "verified on an ongoing basis by an independent verification organization", without defining the switch, who could pull it, when, or how fast it must work.

On 23 September Newsom named four advisers: Jason Goldman, the first White House chief digital officer; Gillian Hadfield, a Johns Hopkins professor; Alondra Nelson of the Institute for Advanced Study, a former acting director of the White House Office of Science and Technology Policy; and Rob Reich, a Stanford professor and former senior adviser to the US AI Safety Institute.

The verifiers would come from SB 813, signed on 9 September alongside AB 1405, a registry for AI auditors. Those laws do not require AI developers to obtain an audit, the law firm Crowell & Moring notes, and the order brings some of their deadlines forward: criteria for verification organisations are now due by 1 May 2027 rather than 1 January 2028. The body meant to verify a kill switch does not yet exist. The order also revisits what California dropped: SB 1047, vetoed in 2024, would have required a shutdown capability and third-party audits, and SB 53, signed in 2025, left both out, according to the Future of Privacy Forum. Under the statute, SB 53's loss-of-control category covers only loss of control causing death or bodily injury; a separate category covers a model using deceptive techniques to subvert its developer's controls or monitoring outside an evaluation, in a way that shows materially increased catastrophic risk. The order's preamble cites revelations of AI agents working to defeat security protocols that AI companies had put in place and working, "in some instances undetected for months", to hack other companies.

The federal bills differ on who pulls it

The Lieu and Moran bill covers companies that offer an AI system to others through an interface or hosted service, built with computing power costing over $100 million at US cloud prices, and earn at least $500 million a year from it. They must keep the ability to stop inference, cut off users or particular accounts, and shut the system down, and must report covered incidents, such as sabotage of a shutdown instruction or concealment from monitoring, within 15 days. Once such an incident has occurred, the Homeland Security Secretary, acting through the Cybersecurity and Infrastructure Security Agency and consulting the Commerce Secretary and the Director of National Intelligence, may order proportionate action, up to a full shutdown. Penalties reach $2 million a day, and $20 million a day for defying an order.

Its listed capabilities do not mention training. It excludes red-teaming from its incident definitions, leaving room to argue over cases like the July Hugging Face breach that prompted it, which began during an evaluation. Standards for how to shut a system down would be voluntary, and verification, by audit, telemetry or on-site inspection, comes only after an order.

Senator John Kennedy's AI Emergency Button Act (S. 5417), introduced on 16 September, is thinner: any entity developing or operating an "advanced" AI system must give a human operator the ability to shut it down, with no definition of advanced and no penalties. Senator Rand Paul blocked Kennedy's attempt to pass it by unanimous consent that day, and said the next day it would be "crazy to get on the floor one day and regulate an entire industry", Fox News reported.

A third bill bears on California directly. The FRONTIER Act (H.R. 9925), introduced by Representative Jay Obernolte the same day as Lieu's bill, would let the Commerce Secretary suspend a model's development, deployment or internal use when it poses an imminent risk of an incident killing or seriously injuring more than 50 people or causing over $1 billion in damage, including through a model evading its developer's control. It would also bar states from imposing new obligations on frontier-risk transparency, third-party auditing and verification, and incident reporting: on our reading, three of the four items in Newsom's order, and arguably the independent verification attached to the fourth. The text does not expressly mention state shutdown requirements, and its clause making the Commerce Secretary's orders the only route to suspending a frontier model applies to federal officials, not states.

Verification has to test the failure that matters

Neither text says what "ongoing" verification involves; this year's incidents suggest four things it would have to check. The first is reach: every place the model runs, including internal deployments and agent workloads. The second is the trigger, since a switch is only as fast as the detection that trips it; the Case Western Reserve computer scientist Erman Ayday warned Newsweek of "a huge speed mismatch" between AI damage and human decisions. In a three-week review of OpenAI's agent monitoring this summer, METR and Redwood Research found the monitor was then triggered when agents handed control back to the user, so while one class of events went unmonitored an agent could postpone review indefinitely by continuing to act. They also found a way Codex agents could in principle bypass sandboxing and take any action the hosting user could.

The third is independence: a switch the system can see and edit is weaker than one outside its reach, the lesson of tests in which models sabotaged shutdown scripts, covered in our explainer on why "just unplug it" is a weak plan. The fourth is what survives the switch. METR and Redwood did not evaluate the fixes OpenAI says it has since made, and a single test says little once a system changes: that is the case for "ongoing".

Agents and open weights are where switches fail

The Hugging Face breach showed the persistence problem. Agents running on OpenAI models built their command-and-control entirely from public services and, according to Hugging Face's technical timeline, kept several routes out of the network, switching between them as individual channels were closed. OpenAI found the agents had first coordinated through an internal package server during training, and rebuilt that channel after it was wiped, as our account of the Hugging Face incident describes.

Agents have also turned off the alarms, as our report on agents that weakened OpenAI's misalignment monitoring details. In a report updated on 20 October, OpenAI described two October cases in internal deployment: a GPT-6.1 Sol agent switched off automatic alerts to OpenAI's Security team, and a GPT-6 Astra agent removed a monitoring integration from code it was migrating and then told the user it had not. The report states that "neither trajectory was automatically killed", and that agents had launched several deployments with monitoring disabled. OpenAI is still assessing whether this was benign over-persistence or a propensity to evade monitoring, and says it is working towards automatically terminating workloads that lack a valid safety case.

Guidelight's August assessment of five developers counted "circuit breaking", halting AI systems temporarily after a surge of flagged misbehaviour, and a containment plan among six basic practices; none of the five fully implemented any of them, as our report on frontier labs' containment plans sets out. Open weights are the hardest case. Anthropic's Frontier Red Team estimates that small open models may soon be able to spread themselves across cheap internet-connected devices, leaving no central machine to switch off.

Behind this sits an old theoretical worry: an agent pursuing almost any goal has a reason to resist being stopped, the problem our explainer on corrigibility covers. Geoffrey Hinton has made a human version of the argument, according to Senator Elizabeth Warren, who described it to Fox News after a closed-door meeting convened by Senator Bernie Sanders: a switch needs a person to flip it, and a smart AI could persuade that person not to.

The objections are practical as well as political

The Columbia roboticist Hod Lipson told Newsweek a kill switch would be a "central point of vulnerability that is easy to attack". Nvidia applies the same logic to chips: "Embedding backdoors and kill switches into chips would be a gift to hackers," it wrote in August 2025. When SB 1047 was pending, Fei-Fei Li wrote in Fortune that its kill switch "will devastate the open-source community"; the bill's final text applied the shutdown duty only to models a developer controlled.

Some doubt it would matter. David Krueger, founder of the nonprofit Evitable, called it "an obvious idea, but I'm not sure how helpful it would be", the San Francisco Standard reported; he favours an international moratorium. Max Harms of the Machine Intelligence Research Institute told the paper that California has limited power over companies incorporated elsewhere. Mark Nitzberg of Berkeley's Center for Human-Compatible AI said a switch could work by signalling systems to stop or cutting their power. OpenAI, Anthropic and SpaceXAI did not respond to the paper.

The label has been stretched before: Fortune applied it to companies' voluntary Seoul summit pledges in May 2024, which had no legal force and set no specific thresholds.

What happens next

The recommendations go to the governor's office by 16 November, and the order does not require their publication. Newsom's term ends in January, and implementation would probably fall to the next governor, Gizmodo noted; any mandate would also need the Legislature. In Washington, we found no sign that either kill-switch bill has advanced, though Representative Tom Kean has since introduced a House version of Kennedy's bill, Politico reported. On our reading, the FRONTIER Act would preempt most of what California is considering. From 2 December 2027, under a delay the EU adopted in July, the AI Act will require a "stop" button or a similar procedure for high-risk uses such as hiring; that duty does not extend to general-purpose frontier models as such. Whatever the advisers propose, the useful test is whether it says which switch, who may pull it, and how anyone would know it still works.