David Robinson formerly authored the safety reports accompanying every major model release at OpenAI. This week, he stepped down from his role and is now voicing his concerns in an editorial in The Atlantic.
It is understandable if you feel a bit cynical about former builders suddenly warning about the dangers of the tools they helped create, since they did, after all, contribute to the situation. But that does not mean we should dismiss their warnings.
Robinson argues that the culture within the industry is fundamentally broken. He believes this is a deeper issue than simply applying a few new rules or regulations to how we handle training models. Silicon Valley has operated with “extreme confidence” and “perpetual sprints,” he says, building bigger and better models with “unimpeded optimism” that ignores or underestimates potential problems.
He says the time has come for AI companies to develop a sense of humility and look outside the insular, move-fast-and-break-things world of the tech industry. Specifically, he says AI needs nuclear-level safeguards:
“Given today’s risks, frontier labs need to run like nuclear power plants or busy airports, with layers of redundancy and careful, time-consuming planning, so that the occasional and inevitable human error does not open a door to disaster.”
Given today’s risks, frontier labs need to run like nuclear power plants or busy airports, with layers of redundancy and careful, time-consuming planning, so that the occasional and inevitable human error does not open a door to disaster.
Robinson is just the latest in a growing parade of researchers and safety workers who have left their positions at prominent AI firms. Jacob Coxon seems to have kicked off the exodus by quitting Anthropic and then publicly saying AI “could kill us all by the end of the decade.”