Anthropic suggests pause in the development of Artificial Intelligence

The North American company revealed in a publication on Thursday that it is increasingly delegating the development of artificial intelligence (AI) to its own AI systems and that, as computational capacity increases, the possibility of so-called “recursive self-improvement” approaches, that is, when the system itself is able to conceive and develop its successor.

This recursive self-improvement, which is “not inevitable” and may arrive “before many institutions are prepared”, could bring benefits to science and health, but also “increase the risks of humans losing control of AI systems”, highlighted the company.

“If systems are capable of entirely creating their own successors, ways to make them secure, supervise them, and shape their behavior become much more important,” he added.

The company believes it will “likely be beneficial” to slow down the development of AI so it can spend more time dealing with its “immense implications,” and has asked companies around the world to collaborate with its research arm, The Anthropic Institute, to establish a framework.

“We believe it would be good for the world to have the option to pause or temporarily pause the development of cutting-edge AI, to allow societal structures and research to align with the pace of advancing technology,” he stated.

Anthropic proposes to help build the systems necessary for this pause, including ways to “verify” whether others actually stop development, avoiding favoring a “malicious agent”, and highlighted that “well-funded” laboratories in several countries should “accept stopping under the same conditions”.

“A credible pause also needs to specify what triggers it, what ends it and who arbitrates it,” he said.

The text, signed by Anthropic co-founder, Jack Clark, and the director of The Anthropic Institute, Marina Favaro, describes based on internal data how quickly its advanced models are evolving and predicts that, in the future, agents like Claude will be able to train themselves.

For example, the company explained that today more than 80% of the code incorporated into its code base is already generated by Claude, while just over a year ago, in February 2025, this number was less than 10%.