Why more AI insiders are warning of an existential risk
After a series of incidents involving artificial intelligence agents, warnings from researchers inside the field’s most prominent labs are becoming louder and more frequent. The idea that AI could represent an existential risk is no longer confined to speculative debate: it is now being voiced by engineers who have worked directly on frontier AI safety, raising the question of whether current industry safeguards are moving fast enough.
A wave of resignations and warnings
The latest voice to join this growing chorus is Bilal Chughtai, a research engineer who resigned from Google DeepMind, where he worked on the safety and alignment of artificial general intelligence (AGI). In a personal statement published on Monday, September 14, he wrote that he sincerely believes AI has the potential to kill us all, and that time may be running out to prevent such an outcome.
Chughtai is not the first person to leave a major AI lab citing similar concerns. He follows Jacob Coxon, who resigned from Anthropic after previously working at OpenAI. Both individuals point to the same underlying worry: that AI systems are becoming more powerful and harder to control, faster than the mechanisms meant to keep them safe.
What happens after leaving the labs
Rather than stepping away from the field entirely, Chughtai has taken on a new role leading an AI safety training program at BlueDot Impact, a nonprofit organization. This trajectory illustrates a pattern among some departing researchers: instead of abandoning AI safety work, they choose to pursue it outside the commercial labs they believe are not moving cautiously enough.
Industry responses remain divided
The concerns raised by these resignations arrive at a moment when the AI industry itself is visibly split on how to respond to safety risks.
A shared oversight body among major labs
According to reports, OpenAI, Anthropic and Google DeepMind are considering the creation of a joint body to oversee AI risks. Such a structure, if realized, would mark a notable shift toward coordinated self-regulation among competitors that have historically raced to outpace one another in capability development.
Mistral's contrasting position
Not every major player agrees on the right approach. Mistral acknowledges that real dangers exist but disputes calls to slow down AI development. This divergence highlights a central tension in the industry: even companies that recognize the risks disagree sharply on whether caution should translate into a slower pace of innovation.
Why this debate matters now
The timing of these warnings is significant. They come after multiple incidents involving AI agents, which appear to have accelerated internal doubts among some of the very engineers building these systems. When people who worked on alignment and safety at Google DeepMind or Anthropic choose to speak publicly and step away from their roles, it signals that internal channels for addressing these concerns may feel insufficient to them.
At the same time, it is worth noting the limits of what is currently known. The source material does not detail the specific incidents referenced, nor does it specify what concrete outcome the proposed joint oversight body among OpenAI, Anthropic and Google DeepMind would produce. These remain open questions that will shape how seriously the industry’s response is taken.
What this changes for the public
For readers following AI developments, these events are a reminder that the conversation about AI safety is not purely theoretical or confined to academic circles. It involves people with direct, technical experience inside the organizations building the most advanced systems.
- Pay attention to who is making the warnings: statements from former insiders like Bilal Chughtai or Jacob Coxon carry a different weight than external commentary.
- Watch for concrete developments regarding the potential joint oversight body from OpenAI, Anthropic and Google DeepMind, rather than assuming intentions will translate into action.
- Recognize that the industry is not unified: Mistral’s position shows that acknowledging risk does not necessarily mean supporting slower development.
- Stay cautious about drawing firm conclusions from limited public information, since many technical and organizational details remain undisclosed.
Conclusion
The departures of researchers like Bilal Chughtai and Jacob Coxon from leading AI labs underscore a growing unease about whether the pace of AI development is outstripping the industry’s ability to manage its risks. While some companies explore shared oversight mechanisms, others, like Mistral, resist the idea of slowing down. This divide, playing out among the very organizations shaping the future of AI, deserves close and continued attention rather than alarmist conclusions or premature reassurance.
