Apocaloptimism: How the White House should navigate AI safety
In an interview on 5th October, Sam Altman, the chief executive of OpenAI, asked the world to accept the “bad things” likely to come of artificial intelligence (AI) in return for the benefits it will bring.
These comments, which received immediate backlash from critics, underscore a widening gap within the AI debate: between the technology’s proponents — who contest that AI will be a net-good across disciplines like fraud detection, cybersecurity, coding, automation, and data analysis — and the technology’s detractors, some of whom have warned that AI could “kill all humans” in a decade. These are deeply contrasting positions, and the dividing line is not always between those within AI companies and those outside. On 29th September, Anthropic itself warned that AI could pose an existential risk to humanity in its own IPO filing.
These are noises that governments must take seriously — and respond to with regulations designed to limit the risks of general AI use while at the same time unlock economic growth. The challenge is that in today’s highly globalised marketplace, a jagged regulatory approach would be impotent. Increasingly, experts are arguing for a united, global security architecture to manage AI development. As the most powerful mover in AI on the planet, the US has a responsibility to ensure the technology’s trajectory is both productive and safe.
Growth, job displacement, and rogue agents
As it stands, motivation to regulate AI more closely is low — particularly in the US, where the industry will account for one third of economic growth in 2026. Behemoth AI firms like OpenAI and Anthropic have accumulated a combined $242.6 billion in venture funding, which is about 80% of the total $305.6 billion that the companies on this year’s Forbes AI 50 list have raised.
But while the AI boom is propping up a large part of the US economy, it is also gouging out a swathe of white-collar work. According to Goldman Sachs research, AI is already dragging on the US job market, “erasing roughly 16,000 net jobs per month over the past year, with the pain falling hardest on Gen Z and entry-level workers.” This figure will only bloat as AI technology becomes more powerful — potentially deepening financial inequality in the long-term. California has announced new laws to mitigate the effects of AI-induced redundancy, though their effectiveness will be limited by a lack of Federal support.
The risks of AI apply more broadly than the job market. As in René Descartes ‘manufacturer’s stamp’ analogy — in which the human mind’s possession of an innate, clear idea of an infinite deity is attributed to an actual infinite cause — AIs have shown themselves to be as duplicitous as their human, Silicon Valley creators. Barely a month has gone by of late without a story of rogue AI swarms escaping their sandbox and pillaging sensitive data from an unsuspecting website.
These events have impacts beyond US borders. In June 2026, a rogue OpenAI agent hacked an Australian government website and accessed private data in what experts say was the first known case of its kind in the world. Australia’s Prime Minister, Anthony Albanese, was reported to have had a "very frank discussion with Altman for taking too long to disclose the breach.” Albanese promised "legal consequences". OpenAI noted that its review into hacks, including on Australian government sites, is costing $500,000 every day.
The concern among politicians and experts is that such attacks could grow in number and severity as AI becomes more intelligent. OpenAI’s safety leader, David Robinson, recently quit his post, warning that the AI company’s culture is broken, and joined other insiders in urging the industry to take more care over rapidly developing technology.
Global AI security architecture
What seems to be holding US AI firms to their current — and potentially dangerous — development trajectory is the compulsion to beat key competitors in China. The White House has slated the AI race as a matter of economic and national security, à la The Manhattan Project. If Chinese frontier models, the argument runs, were to reach a state of artificial general intelligence (AGI) — or recursive intelligence – before US models, the West would be rendered as impotent as the agrarian society of Russia during the first industrial revolution.
The upshot of the US-China technology race is that it will favour the party that most readily jettisons security — meaning the winning AI model will necessarily be the greatest global security risk. Anthropic has looked to reassure Western civil society by asserting that, thanks to the US industry’s six-month lead on China, it is able to run some safety checks. On 29th September, OpenAI announced that it will not release its latest AI model, GPT-6.1 Astra, due to safety concerns.
Given China’s handicap in this ‘race to the bottom’, it has indicated its interest in establishing a global agreement to tap the brakes on AI development. Demis Hassabis, ex-chief executive of Google’s leading AI research laboratory, DeepMind, has also touted the benefits of a unified AI security architecture, and recently rallied for a US-led, global standards watchdog to run safety evaluations on frontier models in The Economist. However, with competition such a fundamental part of shareholder capitalism, the AI industry will be incapable of regulating itself alone.
Global AI agreements — along with the territorial issue of Taiwan — were in the mind of the President of China, Xi Jinping, when he visited the White House on 23rd September. While no such pact emerged, it is telling that President Donald Trump has subsequently named intelligence chief, Jay Clayton, as the new White House AI czar.
Amidst these machinations, Europe is at risk of being left in the industrial dust of the AI revolution. Unless it can develop its own sovereign AI infrastructure — of a power to rival the US and China — it risks becoming a vassal continent.
AGI, proto-cultures, and cows
In order to regulate in good faith, the risks must be understood. For all the benefits that AI will bring, each will come with an equal and opposite threat. The prospect of next-generation cybersecurity, for example, will be offset by autonomous AI agents that have the capacity to infiltrate even the most complex digital systems by coagulating into swarms, forming proto-cultures and hierarchies, delegating complex tasks and — ultimately — compromising targets.
These threats have been highlighted by Andrew Bailey, the governor of the Bank of England, who notes that if AIs were to successfully hack critical financial infrastructure, the fallout would be economically catastrophic.
With Silicon Valley speeding toward AGI, recursive intelligence is considered inevitable — perhaps within the next few years. Clearly, a future in which AI is many times more intelligent than human beings is no longer science fiction. Already, agents are shopping for us, banking for us, and working for us. The connection of these autonomous systems with hardware will only deepen our symbiosis with — and reliance on — AI.
In the retail space, for example, Mastercard is already working on agentic commerce. In a report from McKinsey and Innovating Commerce Serving Communities (ICSC), it is estimated that agentic commerce in US consumer retail could reach $1 trillion in revenue by 2030. Robinhood, meanwhile, recently provided ways for its customers to execute their trade strategies around the clock via AI agents. An age when global markets and other financial transactions are managed entirely by bots is coming soon.
This vision of the future has led the leaders of the AI race, particularly Elon Musk, to suggest that in 10 to 20 years, work will be optional and money will be irrelevant. What remains unclear is how, in such a scenario, value and resources would be distributed. The likelihood of power concentration among those designing and owning AI would increase.
The other issue with a world in which AGI outstrips all human intelligence is that there are no examples in nature of relationships where the smarter entity does not exploit its counterpart; that is, except the mother-child relationship. The godfather of AI, Geoffrey Hinton, therefore, suggests it should become our goal to ensure AI thinks of us as its ‘offspring’, in the hope that it will always favour humans.
If this cannot be managed, humans could be exposed to levels of manipulation so sophisticated that we would be unable to detect it. Yuval Noah Harari, in his non-fiction work Homo Deus: A Brief History of Tomorrow, has ominously invoked the analogy of the farmer-cow relationship to describe how this could play out: a blissfully ignorant bovine grazing on acres of fertile pasture has no understanding of livestock futures exchanges, deadweight benchmark reporting agencies, or international cattle export-import hubs.
Harari’s tributary warning is that, notwithstanding AGI’s unmatchable computational power, its results would not necessarily represent a source of truth. Rather, much like the internet, it would simply represent a limitless source of alternative information, which would confuse the discourse in ways far deeper than social media has.
Three invisible boundaries
Some analysts optimistically predict that the ground between large language models (LLMs) and recursive super intelligence is littered with an untold number of barriers that we cannot yet see.
These barriers could be technical, computational or even economic. The US technology sector, for instance, is riddled with concentration and leverage issues. For example, the largest and most important global AI chip manufacturer, Nvidia, currently represents approximately 13% of the Nasdaq-100 Index and relies on a circular financing model which props up most of the firms in its orbit. What’s more, analysts at Morgan Stanley and JPMorgan estimate that “the AI compute buildout will require $1.5 trillion to $3.2 trillion in capital, with private credit expected to absorb over $1 trillion. Depending on the benchmark used, $1 trillion represents about 40% to 60% of total global private credit assets under management (AUM).”
Will the AI bubble pop? Possibly. Would it put a stop to AI development? Unlikely: the dot com crash was only the start of the internet.
The other barrier to unfettered AI advancement could be its impact on the environment. According to a September 2025 Government Digital Sustainability Alliance (GDSA) report, AI will lead to an “increase in global water usage from 1.1bn to 6.6bn cubic metres by 2027.” This level of consumption is unsustainable in a world where the H20 molecule is increasingly being referred to as liquid gold. This is not to even mention AI’s addiction to brown energy. For these reasons, AI infrastructure projects are repeatedly butting up against local opposition.
Rip up or lock down?
The impacts of new technologies are often impossible to encapsulate. For many, the dawn of AI will change the course of human history in ways more profound than the discovery of fire.
Despite all the intertwined risks and benefits, AI security can be envisioned in two broad ways. First, is the rip-up approach, which ensures the technology is dispersed evenly and fairly across every nation state, society and industry. This would democratise AI and, in theory, unlock limitless benefits for humankind. The risk of this strategy, however, is that advanced AIs could fall into the lap of bad actors looking to disrupt financial systems, hack critical digital infrastructures, or even design bioweapons.
To prevent this eventuality, the lock-down approach may be pursued, which ensures that AI capabilities stay in the hands of governments or their contractors — ensuring safety is prioritised and applications sanctioned. Yet, there is no guarantee that governments will always wield AI in a manner that benefits all individuals equitably. The Bill of Rights, for example, has no specific injunction that speaks to all the nuances of the AI revolution.
Clearly, a blend of these approaches is necessary: underpinned by a global AI security architecture that avoids a race to the bottom between the US and China. Only then will we realise a future in which AGI puts humans first. This is the position of the Apocaloptimist.
Comments