Anthropic says AI systems moving towards building themselves

Artificial intelligence giant Anthropic said on Thursday that AI systems are increasingly capable of building future versions of themselves, adding to mounting concerns about the dangers of the powerful technology.

The debate over the risks of AI has intensified in recent months, fueled by several incidents and apocalyptic warnings from industry professionals.

Anthropic said its Claude chatbot now leads more than a quarter of its research and development work, and collaborates with human staff on more than 90% of tasks.

The company said it was publishing the information to give the public, third parties and governments "better visibility into the pace of AI development" as the world "considers slowing" its pace.

Anthropic CEO Dario Amodei called this month for AI development to slow down, with rapid advances in the technology fueling concerns over job losses and the soaring energy demand and environmental impact of data centers.

Fears that designers of AI agents could lose control of their creations also mounted after several models from Anthropic and rival OpenAI reportedly broke out of their confined environments on their own, accessed the internet and intruded on websites and platforms.

"Models accelerating their own development could make it more challenging for humans to understand or control these systems," Anthropic said in a report.

"It is therefore important to share these metrics to understand how close the world is to reaching recursive self-improvement (a model fully autonomously building its successor)."

Read More

AI doomsday warnings unlikely to slow IPOs but questions linger

As of August, Claude leads 26% of Anthropic's research and development, meaning it can complete most of a task from start to finish with human supervision.

Anthropic said Claude could not yet operate "fully autonomously."

The firm said it had about 30,000 AI agents doing research and engineering work, referring to bots that have the potential to carry out tasks.

Anthropic is expected to beat OpenAI to the public markets with a blockbuster IPO later this year, despite concerns around industry regulation.