Carlos Barria/Reuters
- OpenAI CEO Sam Altman said some AI accidents are unavoidable as models grow more capable.
- He said AI should adopt the FAA and NTSB’s culture of reporting and learning from accidents.
- His remarks come as AI safety researchers warn that frontier labs are “gambling with our lives.”
Sam Altman says some AI accidents are unavoidable, but it’s how you learn from them.
Speaking at Salesforce’s Dreamforce conference on Tuesday, the OpenAI CEO said tech companies should embrace a culture of transparent accident reporting, similar to that of the aviation industry.
“I do think that some degree of accidents are unavoidable with new technology,” Altman said. “What I really care about is that we have got a great culture of accident reporting and learning.”
He cited the Federal Aviation Administration and the National Transportation Safety Board as examples the AI industry should follow, saying the agencies’ emphasis on investigating incidents has helped make flying safer.
“You know, accidents are going to happen,” he said of the FAA’s and NTSB’s approach. “We’re going to learn as much as we can and react from each one.”
Altman’s comments come as the AI industry faces renewed questions about whether its biggest labs are moving too quickly toward powerful autonomous systems.
Jacob Coxon, a researcher who quit Anthropic last week and previously worked at OpenAI, kickstarted a debate by saying that the companies were “racing straight to self-improving superintelligence and gambling with our lives.”
‘The world is right to be afraid of this’
Other AI safety researchers have since told Business Insider that advanced systems could one day evade human control or make cyber, biological, and other catastrophic threats easier to carry out — although they would face hurdles in accessing the physical world.
Altman acknowledged that AI doomsday fears are legitimate. “The world is right to be afraid of this,” he said at Dreamforce, pointing to the possibility of a loss-of-control accident and the risk that a small number of AI developers could concentrate too much power.
He said that the answer is not to abandon the technology, and that AI companies must instead ensure that “alignment and safety and monitoring” remain “way ahead of capabilities.” He added that they should also be prepared “to slow down or stop” if they cannot do so.
His comments may also sharpen a central criticism from AI-safety advocates: that frontier labs say they recognize the stakes yet continue building more capable models.
Altman said “people get very scared” when it seems like commercial pressures might get in the way of companies or countries doing “the right thing.”
The point is not to accept disaster, Altman said, but to disclose incidents, learn from them quickly, and prevent them from escalating.
“Maybe we can avoid, you know, ever getting to those kinds of stakes,” he said.
The debate is also unfolding as the Trump administration has shown little interest in regulating the sector, seeking to maintain the US’s lead over China.
David Sacks, a presidential advisor on science and tech and the former White House AI Czar, reiterated that stance on Sunday, responding to calls from OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei to “pace the frontier.”
“Dario has written that we need to ‘pace the frontier,’ and Sam has agreed. People may be surprised by my response: go ahead,” Sacks wrote on X. “But stop pretending you need anyone else’s permission.”
Â
