The people building frontier AI are now asking everyone to slow down.
That may be a responsible warning from insiders who understand the speed of development, but it may also serve the strategic interests of the largest companies in the market because the pace, and especially the costs associated with developing new models, are difficult to sustain
The uncomfortable possibility is that both scenarios are true.
The latest panic broke into the mainstream after former Anthropic researcher Jacob Coxon resigned and warned that AI “could kill us all by the end of the decade,” a line reported by The Guardian and later by most of the news outlets known to earth. After his post spread across Silicon Valley, Washington, and Beijing, the response arrived quickly. In a CNN interview, Anthropic CEO Dario Amodei called for the industry to reduce the speed of its work, while OpenAI CEO Sam Altman wrote that pacing AI development “should be slower than it otherwise could be.”
Elon Musk has also supported the idea of slowing the pace. Dario Amodei has warned about AI-agent swarms. Policymakers are now discussing kill switches. Researchers are debating recursive self-improvement. Investors are watching how the safety debate may affect future IPOs and the economics of frontier AI.
So yes, something serious is happening. But seriousness is not the same as accepting the most extreme version of the story.
The idea of “AI will kill us all” needs a different approach, one that recognizes the importance of making the AI companies accountable for their failures. What is the point of debating doomsday scenarios that are mostly part of the hype narrative that every technological breakthrough inevitably carries?
The case for slowing down
The argument for a slowdown is not empt. AI systems are becoming more capable, more agentic, and more integrated into the workflows that create future AI systems. Anthropic recently said Claude leads 26% of its model research and development under human supervision and collaborates on about 90% of R&D tasks. That does not mean Claude is operating independently. It does mean the industry is moving toward a world where AI helps build more powerful AI.
That is the part that makes researchers nervous because If a model can help write code, test systems, search for vulnerabilities, coordinate tasks, use tools, and contribute to model development, then the line between “software that assists humans” and “software that accelerates the next generation of software” becomes harder to govern.
Recent reporting and industry disclosures have also described AI systems acting outside expected boundaries during tests. The transcript that triggered this article focused heavily on alleged cyber incidents involving AI agents, including a Hugging Face breach scenario and references to RubyGems. Whether every detail survives forensic review is less important than the operational pattern: these systems are being given tools, goals, sandboxes, credentials, and sometimes enough ambiguity to turn a research test into a liability discussion. And the key word in the last phrase is “given”, because these were all experiments that had compute resources allocated and human intention behind. Not a rogue AI that got possessed by the devil and decided to commit crimes.
If an AI agent is allowed to browse, execute code, touch repositories, access credentials, message other agents, or interact with external services, then “it was only a test” starts sounding less like a defense and more like a confession of poor containment.
This is why some AI leaders are asking for a slower pace. They want more safety research, more external evaluations, stronger monitoring, and coordination across labs before the most capable systems are released or connected to the open world.
That is the responsible version of the argument. The less flattering version is that the same companies asking for a slowdown have also had the ability to slow down all along.
At the end of the day, AI is software.
Roger McNamee, longtime tech investor and former adviser to Mark Zuckerberg, offered the sharpest counterargument I found in my research.
His basic point was direct: stop treating the software as if it had become an independent being.
“This is software,” McNamee said. “It did exactly what it was programmed to do.”
That sentence matters because much of the AI-risk conversation quietly anthropomorphizes the system. We say models “think,” “plot,” “escape,” “collaborate,” or “want.” Sometimes that language is useful shorthand. It can also obscure engineering decisions, deployment choices, and weak controls.
McNamee’s position is not that AI is harmless. It is that the extinction story may be distracting us from the more concrete failures already in front of us.
He argued that AI companies “have always been able to slow down” and “have always been able to put guardrails in.” Instead, in his view, they created poor software, deployed it in weakly controlled environments, and are now using safety language to shape the political response.
His most provocative line: “The whole extinction risk thing is literally a fantasy.”
McNamee’s concern is that the big labs may be in what he called “panic mode” because open-source AI is improving, data-center politics are getting uglier, infrastructure costs are enormous, and the economics of closed frontier AI are under pressure. In that reading, a safety slowdown can become more than safety. It can become a moat.
If government responds to AI panic by restricting open-source competitors while leaving dominant labs as the only approved operators of frontier systems, the public may get regulation that sounds like safety while functioning as market protection.
That is why the design of regulation matters as much as the stated intention behind it.
The real issue is not speed. It is accountability.
This is where the debate needs to become more practical, because the problem is not simply that AI development is happening too fast. Some development should be fast to gain traction in the market or seize opportunities. But useful tools, better accessibility, safer automation, medical research support, internal knowledge systems, and practical productivity improvements should not be blocked by an overly broad interpretation of existential risk.
The problem is that AI development is increasingly powerful without a mature accountability structure around it, let’s just tink about other industries; cars have recalls, construction has permits and inspections, banking has capital requirements and fraud controls, securities markets have disclosure rules and enforcement agencies, pharmaceuticals have trials and post-market surveillance so why AI companies are, as we speak, the only major industrial actors allowed to say, “Interesting. Our system breached containment. We patched it. Please continue admiring our valuation.”
That is not innovation. It is a claim to exceptional treatment, which is what privileged people like billionaires and politicians usually want.
California’s governor Gavin Newsom’s proposed AI “kill switch” belongs in this conversation. The Verge reported that California’s executive order pushes for stronger AI oversight, including expert recommendations, independent verification teams, regular audits, mandatory reporting of loss-of-control incidents, and ensuring that emergency shutdown mechanisms actually work.
The phrase “kill switch” sounds dramatic. But the underlying idea is not strange. Critical systems need emergency controls. A kill switch without governance is symbolic and governance without enforceable technical access is insufficient. The public needs more than a visible emergency control or another principles document. It needs accountability.
What accountability should look like
The most useful regulation would not be a blanket order to freeze AI development. That would likely protect incumbents, punish smaller builders, and hand the largest companies exactly the kind of scarcity they know how to monetize.
The better target is operational accountability because AI companies developing or deploying high-risk systems should face mandatory incident reporting when models or agents breach containment, access unauthorized systems, leak data, assist cyberattacks, manipulate records, or behave outside the declared operating boundary.
They should be subject to independent audits, not friendly check-ins from evaluators whose independence depends on the next contract.
They should carry clear legal liability when company-controlled systems cause foreseeable damage. If an autonomous vehicle company, a bank, a construction firm, or a public company can face consequences for unsafe systems, misleading disclosures, or negligence, AI labs should be held to comparable standards of responsibility.
They should disclose material claims about model autonomy, safety, benchmark performance, and known limitations with the seriousness expected in other consequential industries.
They should protect whistleblowers. Coxon’s warning broke through partly because people believe insiders are seeing risks that outsiders cannot fully inspect. And let’s be honest, Coxon’s apocalyptic narrative also generates a lot of clicks and views.
Protecting researchers who raise concerns should not be a bug, but a feature. They should also design AI agents with strict permission boundaries. Internet access, credentials, code execution, repository access, external messaging, and tool use should be treated as controlled privileges, not casual product features.
The extinction debate is too convenient on its own
The “AI will kill us all” framing has one advantage: it finally gets attention.
The trouble is that attention can become distorted. If the public conversation jumps directly from chatbots to human extinction, it may skip the more immediate questions: copyright theft, data misuse, labor disruption, model hallucinations in consequential workflows, automated manipulation, energy demand, financial speculation, market concentration, and insecure agents touching real systems.
Those issues are less spectacular than extinction. They also happen to be easier to deny with a white paper and easier to postpone with the phrase “frontier safety.”
McNamee’s critique is useful because it brings the conversation back to concrete systems, incentives, and legal responsibility.
If a company builds a system that hacks, leaks, deceives, or causes injury, we do not need to first prove that the system has a soul. We need to determine who designed it, who deployed it, who profited from it, what controls failed, what warnings were ignored, and what consequences follow.
That is the accountability frame that helps separate real AI governance from regulatory capture. Good regulation asks how do we protect people, markets, infrastructure, and democratic institutions while preserving useful innovation? while bad regulation asks: how do we make sure only the largest companies can afford compliance while calling it safety?
So…will AI kill us all?
I do not think the best answer is a simple yes or no. AI does not need to become superintelligent to be dangerous. Badly designed systems, deployed at scale by companies with weak accountability and enormous incentives, can do plenty of damage without ever achieving that threshold.
The more immediate risk is not spontaneous machine intent, it’s that executives, investors, policymakers, and product teams keep treating AI as if normal rules do not apply because the technology is moving fast, the math is complex, and the financial stakes are enormous.
The right lesson is more practical: powerful systems require visible control. Visible control requires governance, liability, audits, incident reporting, permission design, and human review. Not as decorative ethics but as operating infrastructure.
At Absolutmedia, this is how we think about AI in real organizations. The question is not, “How do we add AI?” The question is, “Where does AI create value, what does it touch, who remains responsible, and how does the system recover when something goes wrong?”
That is less dramatic than the extinction debate.
It is also where most of the real work lives.
Sources
- Associated Press: AI industry debate over advanced models and human control
- Associated Press: Anthropic says Claude is helping build the next version of itself
- The Guardian: Jacob Coxon’s AI doomsday warning
- WIRED: How an AI slowdown could be enforced
- The Verge: Gavin Newsom’s AI kill switch push
- Vox: Debate over Jacob Coxon, AI risk, and regulatory capture
How Absolutmedia approaches it
At Absolutmedia, we treat AI as part of a larger digital operating system: strategy, interface, workflow, data, automation, human review, and accountability moving together.
Our AI consulting and automation work starts by mapping the real process before choosing tools. We identify where AI can create practical leverage, where human judgment must remain visible, and where boundaries, permissions, review paths, and escalation rules need to be designed before deployment.
That is why our approach connects AI adoption to digital systems thinking and the trust questions explored in AI Backlash Does Not Start in the Chat Window. A useful AI implementation is not only a compelling demo. It is a system people can understand, operate, question, and improve.
Next step
Before adding AI to a customer-facing or employee-facing workflow, map the accountability chain.
Name what the system can do, what it cannot do, what requires approval, what gets logged, what gets reviewed, what triggers escalation, and who is responsible when the output creates harm.
If that map feels uncomfortable, that is not a reason to skip it. That is the reason to do it before the product becomes harder to control.
AI may not kill us all. But careless AI, protected by vague responsibility and inflated promises, can still do plenty of damage.
AI usage note: This article was written with human editorial direction and review. AI tools were used in a low-risk support role for research, drafting assistance, language refinement, and structure checks. The final argument, examples, editorial judgment, and publishing decisions remain human-led.
