Jacob Coxon: Anthropic researcher quits saying AI developers believe 'it could kill us all'

Anthropic researcher Jacob Coxon has resigned from the AI company, warning that some people developing advanced artificial intelligence genuinely believe the technology could pose an existential threat to humanity within the next decade.

Coxon, who spent the past three years conducting pre-training research at Anthropic and OpenAI, said neither company was acting responsibly in the race to develop increasingly advanced AI systems.

“They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon wrote on X.

Superintelligence refers to a potential stage of AI development in which machines outperform humans across virtually every field.

Anthropic Alignment Science lead Evan Hubinger responded to Coxon’s comments on X, agreeing that the risk is real. He said he personally estimated the chance of AI killing all humans within the next decade at more than 10%.

Hubinger added that while he believed Anthropic was making serious efforts to address the risks, the company did not yet have a clear solution for aligning superintelligent AI with human interests or a clear path toward achieving it.

Coxon is among a growing number of AI experts calling for greater caution in the development of advanced AI.

In July, more than 1,300 employees from leading AI companies signed a statement warning of a “real risk” that AI capabilities could advance faster than researchers’ ability to understand or control the systems.

The statement urged the US government to support international efforts to develop safeguards that could deliberately slow the pace of frontier AI development.