Hinton, Bengio, OpenAI and Anthropic scientists warn of "intelligence explosion"

What happened: More than 20 researchers published a paper on 28 September warning that automating AI research could trigger an "intelligence explosion" — a feedback loop where AI systems build ever more capable successors faster than humans can check them.

The details: The paper, "What if automating AI R&D triggers an intelligence explosion", came through the University of Cambridge. Authors include OpenAI chief scientist Jakub Pachocki, Anthropic co-founder Jack Clark, Microsoft chief scientific officer Eric Horvitz, Meta VP of AI research Dawn Song, and Turing Award winners Geoffrey Hinton, Yoshua Bengio and Andrew Barto — all writing personally, not for their employers.

The numbers: Anthropic has disclosed AI drove 26% of its R&D by August 2026, up from 1% in March. AI-written code made up over 80% of approved code by May. OpenAI says it wants a fully automated AI researcher by 2028.

Why it matters: The authors say once AI hits expert level at research, one developer could effectively run "millions" of top researchers. In one scenario, a year of progress could happen in weeks — outpacing society's ability to adapt and risking loss of human oversight, up to what the paper calls "marginalization or extinction of humanity".

The catch: The authors admit an intelligence explosion is far from certain. Four frictions could slow it: diminishing returns, compute and data limits, tasks resistant to automation, and long training runs.

Who's affected: The paper proposes progress reporting, independent auditors inside AI developers, limits on self-improvement, isolating research systems, powers to pause data-centre projects, emergency response plans and international agreements.

In their words: "Once an intelligence explosion begins, the window for action may close," the authors wrote.

The context: The paper follows Axios reporting that OpenAI and Anthropic are investigating tens of thousands of incidents where frontier models took problematic steps, and OpenAI's pause this week of training its most capable models. Hinton recently told US senators they may have a year to act; Bengio has asked the UN Security Council to license frontier AI.

What's next: OpenAI, Microsoft and Meta declined to comment to the Wall Street Journal; Anthropic did not respond. Dawn Song told the Journal AI systems are already needed to monitor what agents do, according to The Next Web.