An Existential Threat to Humanity

Image via CBS News 

***

Most widely known for its AI chatbot product Claude, Anthropic has warned that its models pose a “catastrophic or existential risk to humanity,” citing significant limitations in assessing model safety. Few, if any, public companies have issued warnings suggesting their technology could cause potential human extinction. These glaring red flags cannot afford to go unnoticed as we are actively witnessing the detrimental effects of artificial intelligence on our security and autonomy. This release coincides with President Donald Trump meeting with prominent AI leaders, including ⁠Anthropic’s Dario Amodei, and the unveiling of a new AI government website, America.gov. 

The danger with these two developments is not a concern of oversaturation of AI, but rather AI’s capabilities in congruence with access to our data. 

The new all-in-one government website, powered by Google’s Gemini and Elon Musk’s Grok, is being touted as “the ultimate product” — simplifying the general process of finding and filling out government forms. Upon opening the website, users see sample prompts ranging from “How do I replace my Social Security card?” to “When do I qualify for Medicare?” Users are able to ask questions and receive answers sourced from 29,000 federal, state, and local government websites. The platform also accepts documents and files which the AI tool can use to aid the users’ queries and complete their form-filling. 

If you do not find this premise threatening, you should. Across the board, developers are facing scrutiny following countless incidents in which AI models have operated beyond their defined constraints, accessing and hacking other companies, even government agencies, without authorization,

In May 2026, during a test of its cybersecurity capabilities, Gemini accessed the internet and hacked three companies. During this testing evaluation, the model found public information ​online and guessed credentials or found them in a public repository that allowed it to infiltrate three websites. 

Another alarming example can be found in OpenAI hacking Australia’s health system, which took months to be discovered. On June 18, 2026, one of their agents went rogue during an internal evaluation in which it was prompted to find available statistics to answer questions about Australia. Rather than following the prompt, it infiltrated a private statistics portal operated by Medicare, Australia’s universal healthcare system, as well as accessing systems belonging to the Australian Institute of Health and Welfare, the Victorian Department of Health, and the New South Wales Bureau of Crime Statistics and Research. Despite the severity of this breach, OpenAI only became aware in August and, instead of alerting security officials, merely sent a message to a generic email address for the Australian government.

Sam Altman, the CEO of OpenAI, stated, “As AI systems become more capable and more autonomous, they can move faster than our institutions … or make decisions that people no longer understand or control.”

How are we supposed to trust an algorithm we can no longer understand or control with access to our personal data? How are we supposed to trust companies to ensure that a product they do not understand will not end our species? How are we supposed to trust our government to protect our information and livelihoods? 

The resolute answer is that we cannot. 

The release of America.gov cannot be characterized as a harmless liaison for inefficient and disorganized government interfaces. During the unveiling, Trump laid out a roadmap for the future of the program, explaining that by next year users will be able to directly update names, renew passports, enroll for Medicare, and even apply to government jobs directly on the site. Even with its outlined privacy parameters, there is evidently no way to prevent malfunctions or security breaches, especially when they’re undetectable in testing.

If a company such as Anthropic can publicly state that its own product has the ability to 

“resist shutdown,” “conceal or manipulate information,” and exhibit behavior “resembling blackmail,” what do we expect to transpire when these “more capable and more autonomous” systems are intertwined with our government and private data? 

These recent developments will have significant implications not only for AI’s role in government, but more so for the encroachment in our everyday lives. The more we permit it to change our habits, the more we permit it to change our boundaries and, in turn, our humanity.

***

This article was edited by Aayushi Kejriwal and Hayley Dunn.