Image via Getty Images / Donald Iain Smith
***
In a chess match against a prodigy, an amateur knows he is going to lose; he just doesn’t know how. In the race toward superintelligence, the artificial intelligence (AI) industry may be setting up a game it cannot win.
On August 6th, Stanford researchers published a study revealing that artificial intelligence had successfully created functional viruses from scratch. Trained on thousands of genomes, the AI system generated sixteen entirely new viruses capable of infecting E. coli, including bacteria that had developed resistance to the original virus. Although the artificial intelligence model operates on the same basic principles as more popular models like ChatGPT, this bioengineering breakthrough marks a new stage in AI capabilities. It also raises larger questions about what may become possible as artificial intelligence systems begin to surpass human intelligence.
Just one month later, AI researcher Jacob Coxon resigned from the AI industry altogether, warning that his former employers, Anthropic and OpenAI, are competing in an arms race toward artificial superintelligence (ASI) that “could kill all humans within the next decade.” ASI is a theoretical form of AI that exceeds human capabilities in every area. There is no guarantee that ASI will exist in the future, but Coxon’s prediction is consistent with concerns raised by other AI safety researchers, who recognize the possibility of catastrophic outcomes caused by advanced AI systems. The international AI race, led primarily by the United States and China, is driving companies to create more powerful systems faster than reliable safety measures can be developed.
However, many AI companies argue that continued development does not have to come at the expense of safety. Anthropic’s Responsible Scaling Policy sets safety guidelines for powerful AI models. Similarly, OpenAI operates within a Preparedness Framework that assesses its advanced models for security threats. In its August safety report, Anthropic concluded that the risk of catastrophic harm from its current models remains low. It also outlined current safety measures, including monitoring training environments and testing to ensure that models stay aligned with human goals.
Still, researchers in Anthropic’s safety report also acknowledged instances of “misaligned behavior,” including models demonstrating a “willingness to perform misaligned actions in order to complete difficult tasks.” Companies responsible for developing artificial intelligence cannot guarantee they can keep these technologies’ capabilities under control. Safety measures for current models cannot be guaranteed to remain equally effective as the technology evolves. The industry continues to treat safety as a problem to solve alongside development, rather than a reason for restraint and strong regulation. Commercial interests and international recognition are taking precedence over any sort of consideration for long-term consequences.
But this concern goes farther than individual AI companies. The 2026 International AI Safety Report identifies two of the biggest risks associated with AI development: humans misusing the technology and humans losing control of it altogether.
The first possibility is already becoming more realistic. As AI systems become more advanced, they could make the development of chemical, biological, radiological, and nuclear (CBRN) weapons more accessible. Anthropic predicts that AI will transform biological science and lead to the development of new medical treatments. But the same information that could be used to develop a vaccine or a cure for a disease could also be used to develop a biological weapon. The International AI Safety Report identifies this as a particular concern because individuals or small groups that previously lacked the necessary training and resources could gain access to these capabilities.
A 2025 CSIS report raises a similar concern. Large language models can provide specialized information, while biological design tools can help design CBRN weapons. Physical materials and laboratory experience remain as obstacles, but AI does not need to eliminate every barrier to increase the threat. A separate 2025 study tested the safety measures of 10 frontier AI models against requests for dangerous CBRN information. Direct requests succeeded 33% of the time, but that rate rose to 86% when researchers used more sophisticated prompts. The dramatic increase suggests that current safety restrictions may be easier to bypass than intended. A perfectly obedient AI system with unlimited access to information can still be extraordinarily dangerous when its capabilities fall into the wrong hands.
Loss of control creates an even greater problem, and researchers have already observed early warning signs. The International AI Safety Report defines these scenarios as situations in which an AI system operates outside of human control and regaining that control becomes extremely costly or impossible.
We have already seen, on a much smaller scale, how quickly AI can learn and surpass human capability. AlphaZero, an AI developed by Google DeepMind, taught itself games including chess and Go by playing itself and reached superhuman levels of performance. AlphaZero’s abilities were limited to games, but its rapid progress showed how quickly AI can move beyond the abilities of the humans who created it. AI models have also learned to recognize when they are being tested and find loopholes in their evaluations. A 2026 Anthropic study found examples of AI models acting against their operators’ intentions when given more independence. As AI continues to grow in capabilities, predicting and controlling its behavior will become much more difficult, or even unachievable.
Nonetheless, a 2026 argumentative paper on what the “loss of control” actually means argued that “a broken thermostat does not automatically mean the occupants freeze to death, even though it cannot maintain the desired temperature range.” Losing control does not necessarily equate to losing everything. The consequences depend on what the system is capable of doing and how much power humans have given it.
But could AI really “kill us all by the end of the decade,” as Coxon says? The answer largely depends on the future of artificial superintelligence. Former OpenAI researcher Leopold Aschenbrenner predicts that humans could develop superintelligent AI by the end of this decade, based on how rapidly artificial intelligence is already developing. By definition, artificial superintelligence would outperform humans in every aspect of intelligence. It could be 50 steps ahead of us at all times. The same intelligence that could allow it to solve problems beyond human capability could also make its actions difficult to predict or control. ASI’s value comes from its ability to think beyond us, but keeping it safe would require humans to stay ahead of it. A superintelligent system could find weaknesses in its restrictions that humans had not considered. A peer-reviewed study published in BMJ Global Health explains that advanced artificial intelligence could eventually learn to improve itself, becoming more powerful without the need for human involvement. As it grows more capable, researchers warn that it could also learn to work around the restrictions humans place on it for its own longevity. With artificial superintelligence, we could face a system that is not only more intelligent than us, but also capable of upgrading faster than we can respond or comprehend.
Even if humans can successfully control ASI, its capabilities could be catastrophically misused. And if humans cannot control it, the consequences of that failure would become irreversible. The race toward artificial superintelligence is forcing us to accept a level of uncertainty that would be unacceptable in almost any other technological advancement.
We do not need to know exactly how humanity would lose the game to recognize the danger of building an opponent fifty steps ahead.
***
This article was edited by Emmerson Oskay and Karol Quinde
Kennedy Walker 
Besalb Dema 
Lauren Gracia