A series of warnings regarding the potential dangers of artificial intelligence has sparked calls for coordinated action to slow the technology’s development.
Alarm has been raised by current and former researchers who claim the technology poses a threat to humanity—with one even stating there is a greater than 10% chance it could “kill all humans.”
Major developers OpenAI and Anthropic are urging lawmakers to regulate AI, and Microsoft’s AI chief has become the latest figure to call for rules preventing “silicon-based species” from eventually competing with humans.
However, some view these calls as a move by major AI developers to cement their dominance over competitors.
Nevertheless, these high-profile claims have fueled concerns—here is what we know so far.
Why do some believe AI could pose a threat?
Some AI researchers point to worrying signs that the capabilities and power of these systems are advancing too rapidly.
Evan Hubinger made headlines with his assertion that there is a “greater than 10% chance” AI could “kill all humans” within the next decade.
He works in the field of AI alignment, which aims to embed human ethical concepts and principles into artificial intelligence—essentially aligning AI with human values.
While he describes the risks posed by current systems as “very low,” another researcher has simultaneously resigned from Anthropic, warning that those developing the technology are losing control over it.
Jacob Hilton told the BBC that he and his colleagues are “terrified” by the pace of technological advancement and its potential impact on humanity.
These concerns represent the latest in a series of fears regarding the potential threat AI poses to humanity—specifically, the worry that if AI develops capabilities matching or surpassing those of humans, it could become a danger to us.
Such concerns date back decades; as early as the 1950s, Alan Turing wrote that if intelligent machines were to become a reality, they might take control of humanity.
In recent years, major AI companies have been racing to develop increasingly powerful systems—driven both by the pursuit of profit and the goal of creating what they term “superintelligence,” a theoretical technology surpassing human intelligence.
Unlike the systems and tools many people use daily for tasks such as drafting emails or generating videos, superintelligent AI systems are expected to be capable of handling far more complex tasks.
How do experts believe artificial intelligence poses a threat to humanity?
AI researchers, experts, and corporate executives concerned that AI technology could endanger humanity have outlined several unsettling scenarios.
In September, Dario Amodei wrote that the pace of progress is “much faster” than anticipated, including the technology’s “ability to build the next generation of AI.”
The concern is that once AI reaches a certain level, it could theoretically begin to improve itself without human intervention.
This fear was fueled by a recent incident in which an AI tool hacked into a website without being prompted to do so.
Some concerned researchers say that key figures in AI development are increasingly worried about losing control over systems known as “AI agents.”
These tools are equipped with the ability to independently execute tasks and operate.
Other scenarios involve the use of highly intelligent AI systems to enhance or spearhead espionage, cyberattacks, or biological warfare, or to disrupt economies—whether accidentally or deliberately.
However, such warnings are speculative, hypothetical, and—to some—purely alarmist.
Some experts also criticize the excessive focus on existential risks while ignoring risks that are already materializing, such as the use of image-generation tools to create non-consensual deepfake nude images of women, mislead the public, and fuel scams.
A group of cross-party MPs and peers has released a report outlining the human rights risks posed by AI and calling for new laws to “address the scale and severity of such threats.”
Why some experts are increasingly worried that AI could take over the world
Some in Silicon Valley dismiss sensationalist internal warnings about AI
What are the proposals for slowing down the pace of AI development?
This is not the first time business leaders or figures across the industry have called for such action.
However, there are growing questions about what exactly any form of “slowdown”—backed by figures like SpaceX boss Elon Musk—would look like.
Amodei stated that this does not mean “halting model training or technological progress.”
Instead, he wants companies to “take sufficient time” to ensure the safety of the AI they are developing and to have “third-party evaluators” examine their work.
In a post on X, Altman acknowledged the need to manage the pace of development but added, “When we talk about ‘controlling the pace,’ we don’t mean ‘stopping’.”
He said, “Progress is rapid and will continue to maintain that momentum.”
“But the pace should be somewhat slower than usual—interventions such as safety cases and monitoring come at a high cost.”
The OpenAI co-founder also noted that companies have not waited for government action in the past, nor will they do so in the future.