Former artificial intelligence (AI) researcher Jacob Coxon, who resigned from Anthropic in September, accusing it of playing with human lives, told AFP that he considers the development of powerful technology by giant commercial corporations "insane."
The 27-year-old Briton spoke on Thursday outside Anthropic's headquarters in San Francisco, where protesters demanded a ban on the development of models capable of autonomously creating new generations of themselves. In the industry this is called recursive self-improvement (RSI) - a process in which AI improves its own algorithms without human involvement.
AI used to be developed by small non-profit organizations and academic research groups, but the industry is now dominated by trillion-dollar corporations such as Anthropic and its competitor OpenAI, the creator of ChatGPT.
On Wednesday Anthropic held an initial public offering (IPO) - the company's listing on a stock exchange to raise capital. In the IPO the company was valued at 1.5 trillion dollars, and after the first day of trading on the Nasdaq exchange its value stood at 1.3 trillion. OpenAI was reportedly valued at 1.25 trillion dollars this month as part of private financing.
"I think it symbolizes how insane the situation has become. It really is insane that this technology is being built by commercial companies," Coxon said.
A Cambridge University graduate, he worked for three years at OpenAI on model pretraining - the stage at which they absorb huge volumes of data - and then for several months at Anthropic.
His departure, which he announced on social network X on September 8, made headlines in world media. He accused both companies of a "race directly toward self-improving superintelligence."
"The people building AI sincerely believe it could kill us all by the end of the decade," he wrote.
In his view, such an apocalyptic scenario could theoretically be caused by RSI. "Ultimately you potentially compress decades of AI research into months, and if we don't understand AI, this could lead to the creation of potentially very dangerous, very advanced AI systems by the end of 2027," Coxon told AFP on Thursday.
He called on Anthropic to abandon RSI and accused the company of plans to start it as early as next year.
Coxon resigned before his shares fully vested, but retains a stake in OpenAI that could be worth a small fortune if the company goes public in 2027, as planned. In November he called his stake "blood money," but then pledged to donate its entire value to AI safety organizations, without specifying which ones.
He now plans to work at a non-profit organization, possibly in the field of AI safety research. "Ideally I would like to do safety research, but I think there isn't enough time for that," he said.
"The only thing that will slow companies down is political action. This is now a political problem, not a technical one. We need to slow them down," Coxon said.
He supports the technology in general but urges people to treat it critically. "It's a useful technology, but remember that chatbots are not the final form. It is moving toward very capable broad systems that will perform all aspects of human labor," he said.
More than two months after his resignation, the media buzz around him is taking its toll. He admits that he sometimes stares into space, but feels "okay." "I was very surprised by all of this, but I am more afraid for the state of the world than I feel turbulence in my personal life," he said.