Story
Oct 21, 2026
Anthropic’s safety bill offers a brake on AI — but critics see industry at the wheel
Anthropic is presenting a federal framework for stopping genuinely catastrophic AI failures, while critics question whether companies should set the standards that govern them and whether the proposal could spread far beyond the largest labs.
Anthropic has quietly taken its AI-safety campaign to Capitol Hill with a draft bill that would create a narrow but consequential federal path to block dangerous frontier models.
The proposal would allow intervention only when a model poses a “foreseeable, imminent risk” of killing or seriously injuring more than 50 people, or causing more than $1 billion in damage. The threat would have to involve a biological weapon, a model escaping its developer’s controls, or a cyberattack.1
Anthropic says the draft puts into legislation the principles it has pressed with federal and state lawmakers since June. “These are sensible policies we believe Congress should act on,” a company spokesperson said.1
Under the timeline envisioned by the bill, Commerce-registered independent evaluators would periodically assess labs and their models. If they found an imminent enumerated risk, they would notify the Commerce Department within 72 hours. Commerce could then ask a federal judge in Washington to suspend a model’s release — but no other federal agency could unilaterally stop its development, deployment or internal use.1
That design is the central fault line. One critique argues the leaked framework departs from the FAA-style regulator model previously floated by Anthropic CEO Dario Amodei: labs would write the standards they must meet, while AI-safety advocates want government to write them instead.2 Another warns that evaluators may be paid by the companies they scrutinize, even if payments cannot be tied to outcomes — an arrangement that leaves the appearance of independence unresolved.3
The proposal also arrives amid a wider congressional scramble over catastrophic AI risk and state-law preemption. It mirrors thresholds in California’s SB 53, while requiring safety frameworks, transparency reports, whistleblower protections and incident reporting.1
David Sacks, however, argued the bill’s logic could outrun its stated target. Some provisions might be reasonable for “true frontier companies,” he wrote, but warned that the draft could eventually encompass much of Silicon Valley and damage the startup ecosystem.
4
The result is a familiar Washington bargain: Anthropic is offering government a brake, but seeking a system in which the industry still helps define when it should be pulled.