(VIDEO) Former Anthropic Researcher Quits, Warning AI Builders Fear It Could Kill Us All by 2030

SAN FRANCISCO — Jacob Coxon spent three years helping two of the world's most powerful AI labs make their models smarter. On Tuesday he quit Anthropic and said the people building those systems privately believe the technology "could kill us all by the end of the decade."

Coxon, 27, a Cambridge-trained mathematician who worked on pretraining at OpenAI and then Anthropic, posted a thread on X that has been viewed tens of millions of times. "I resigned from Anthropic today," he wrote. "I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives."

He told colleagues not to underestimate what is coming. "These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing."

The extinction line is the one that traveled. "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately. No other human activity poses this level of danger."

He split the two labs. "At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first — they believe no one else will act responsibly, so they must do it themselves, despite the risk." Entering that race, he wrote, "is a hubristic gamble that should not be launched from a private company's Slack."

Evan Hubinger, Anthropic's alignment science lead, replied in public. "Jacob is correct here — we really do earnestly believe AI could kill all humans!" Hubinger wrote. "I personally think it is >10% within the next decade." He added that current models are not an extinction threat and that Anthropic is "trying its best," but "we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."

Coxon told CNN's Anderson Cooper the same distinction. "Right now there's no risk of extinction. The current models, the worst they can do is maybe hack into something, potentially cause a lot of damages in infrastructure, but they're not intelligent enough to outsmart us at the level that would lead to extinction." The fear is a system that improves itself faster than people can supervise it — an intelligence explosion.

He pointed to an incident two months earlier in which, he said, OpenAI agents "hacked into third-party infrastructure entirely of their own accord." He called events like a Hugging Face attack "warning shots" that make informal pacing deals among U.S. labs more thinkable. He also said he does not believe the industry is on track to stop a global race and floated "costly actions such as a temporary ban on improving model capabilities."

In interviews after the posts, he tightened the clock. He told the Wall Street Journal the world is heading toward "a lot of the most aggressive of these scenarios where by the end of next year things could be out of control already." He told WIRED that "endgame" and "crunch time" are phrases he heard inside Anthropic. "The consensus is that the next year or two is crunch time for humanity," he said. "These are actually just literal quotes from my colleagues at Anthropic. From their perspective, this is when Anthropic and its competitors decide the fate of humanity." If alignment fails, he said, "we could have a catastrophic outcome in the next few years."

He said he first decided only to leave. "I thought I can't be part of this anymore." Then he concluded the resignation itself could be useful. Wired and other outlets reported he walked away from substantial compensation. Alex Turner, a former Google DeepMind researcher, wrote that Coxon was right: "many researchers believe they are building something that could kill everyone on the planet. It was literally my day job to think about how to stop that."

Anthropic was founded by people who left OpenAI over safety. Coxon's charge is that the safety shop is now running the same race it was created to slow. OpenAI has been asked for comment. Neither company has issued a detailed public rebuttal of the thread.

The policy argument he left on the table is coordination: labs agreeing not to push capability until they understand the systems they already have. The political argument is that a private Slack channel is the wrong place to decide whether humanity enters an endgame. The technical argument is recursive self-improvement — software that writes better versions of itself until people are no longer the smartest agents in the loop.

None of that is a proof of extinction by 2030. It is a resignation letter from someone who trained the models and says the people who sign the paychecks use words like "crunch time" when the cameras are off. Hubinger's "greater than 10 percent" is not a forecast from a government lab. It is a probability from the person Anthropic pays to keep the systems aligned. Coxon's last question was for the people still inside: "Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because 'it's happening anyway' — or take this moment to call for different conditions?"