Anthropic says a key AI safety alarm has been triggered, days before its IPO

Anthropic says it has crossed a safety threshold in its Responsible Scaling Policy because AI is accelerating the company’s own research and development. The trigger is reached when AI contributes enough to roughly double the rate at which models improve - compressing two years of progress, at the pace seen from 2018 to 2024, into one. The company based its determination on expected future models and its overall progress, not solely on current systems. An independent AI-testing nonprofit estimated that Anthropic’s models have been improving nearly twice as fast since October 2025, and said the threshold may plausibly have been crossed by mid-July. Crossing it calls for stronger safeguards, not a halt to development. Anthropic says it does not expect Claude to build successive generations of frontier models without meaningful human oversight, but warns that growing reliance on AI could make some processes harder for people to understand and control.

The announcement coincides with Anthropic setting an IPO price range of $70 to $78 per share, implying a potential valuation of about $1.4 trillion, and beginning investor meetings. Nvidia and Google plan to invest $10 billion each at the IPO price as part of a $26 billion private placement involving several other major technology companies. The prospectus also warns that no insurer, or even the insurance industry as a whole, might be able to cover losses from a catastrophic AI incident. Anthropic says it is still assessing what the safety threshold requires and plans further reports. Its filing also increasingly uses “superintelligence” instead of “AI,” following a federal directive, while treating the terms as interchangeable.