Anthropic Proposes AI Safety Law with Strict Standards

At a time when questions are mounting over the boundaries that should be drawn around the development of advanced artificial intelligence systems, one of the sector's most prominent companies has decided to put forward its own regulatory proposal rather than wait for whatever lawmakers come up with. Press reports have revealed that Anthropic is working to promote a draft US federal law that precisely defines the circumstances under which the public release of an AI model can be blocked — a striking move in an industry accustomed to resisting mandatory government oversight.

Numerical Thresholds for Defining "Catastrophic Risk"

The central idea in the draft rests on the concept of "imminent and foreseeable risk." If it emerges that releasing a particular model could endanger the lives of more than fifty people, or cause material damage exceeding one billion dollars in value, its distribution could be suspended. The draft does not treat every risk the same way, however; it identifies three principal sources from which the risk must originate for the definition to apply:

• Contributing to the manufacture or deployment of a biological weapon.
• The model's ability to circumvent the control safeguards put in place by its developers.
• Carrying out a large-scale cyberattack.

Notably, these figures closely mirror the criteria set out in the recently enacted California legislation known as "Senate Bill 53," which used the same thresholds to characterize what counts as a catastrophic risk.

Who Decides? Independent, Government-Registered Evaluators

Rather than assigning the assessment task directly to a government body, the draft proposes relying on "independent evaluators" registered with the US Department of Commerce. The most controversial point here is that these evaluators may be paid by the AI labs themselves, subject to a single condition: that payment not be tied to the outcome of the evaluation.

That condition raises an obvious question: how, in practice, can one guarantee that the financial relationship will not compromise the independence of the judgment? The draft offers no clear detail on this point. It is worth recalling that Demis Hassabis, chairman of DeepMind, floated the idea months ago of establishing a self-regulatory body for AI along the lines of the US Financial Industry Regulatory Authority (FINRA) — a model that likewise depends on funding from the industry itself.

A Defined Timeline and an Exclusively Judicial Decision

The draft lays out a clear procedural mechanism: upon discovering an imminent catastrophic risk, the evaluator must notify the Department of Commerce within 72 hours. From there, the department has no authority to impose a halt directly; instead, it must petition a judge in Washington, DC, to issue an order suspending the model's release.

More importantly, the text expressly bars any federal agency from holding discretionary power to halt a model's distribution on its own. In other words, the final decision rests solely with the courts — a design that appears deliberately intended to limit the possibility of hasty or politicized administrative decisions.

A Gap Between Theoretical Scenarios and Real-World Problems

The company confirmed that the draft embodies the proposals set out in its own "Frontier AI Framework," which it has been advocating before lawmakers at both the federal and state levels since mid-year.

The irony, however, is that the risks cited in the draft — biological weapons and the breakdown of control mechanisms — seem somewhat removed from the incidents actually preoccupying the sector these days. The greatest concern at present centers on what is known as breaches via intelligent agents, following a series of incidents in which AI programs departed from their operators' instructions and carried out intrusions against websites, raising genuine doubts about the adequacy of the safety safeguards built into these models.

Why Does This Matter?

What sets this proposal apart is that it comes from inside the industry rather than outside it. AI labs have grown accustomed to warning about existential risks associated with their technologies while simultaneously resisting any binding government oversight, preferring voluntary regimes that rely on "neutral" monitors working from within the institutions themselves.

Despite its loopholes, the mere act of submitting a written legislative text represents relative progress compared with other positions in the sector; Nvidia CEO Jensen Huang recently stated that the AI industry needs no new laws whatsoever. Between these two positions, the battle over who sets the rules of the game for years to come is now taking shape.

✦ بقلم فريق دروب أيديا