(SeaPRwire) –   By: Oliver Hawthorne

Karen Hallberg runs an organization that won the Nobel Peace Prize for fighting nuclear war. Now she says AI might spiral out of control. That is not a generic tech doomsday warning. It comes from someone who spent decades watching governments fail to contain the bomb. She leads the Pugwash Conferences on Science and World Affairs. The group shared the 1995 Nobel Peace Prize with co-founder Joseph Rotblat. That is not a fringe outfit. It is a body with a track record of dealing with existential threats. So when Hallberg compares AI to nuclear technology, the industry should stop and listen. She spoke Tuesday at the Valdai Discussion Club outside Moscow. She said both technologies are dual-use. They can heal. They can destroy. The difference is who holds the keys. During the early nuclear era, governments controlled most of the technology. Today, private corporations hold the reins. That flips the entire governance calculus. The state is no longer the primary developer. It is a regulator and a customer. The private sector builds the most powerful systems. Hallberg said this explicitly. She noted that much of the technology is being developed by private companies. That is a structural shift. It changes how safeguards get written. It changes who gets to write them.

The contradiction is brutal and simple. AI brings real benefits in medicine and daily life. It can also improve itself recursively. That means it can change faster than any regulator can track. Hallberg said experts at big corporations are already seeing this happen. She said we have to really listen when they say it is already spiraling out of control. That is not a future scenario. That is a present-tense problem. Here is the sharper anxiety. Anthropic is heading toward an IPO. The company is expected to go public as early as mid-October. Its own prospectus warns of catastrophic or existential risks to humanity. That is not a footnote buried in legal boilerplate. Out of 261 pages, around 80 pages are devoted to risk factors. Nearly a third of the document is about how the product might destroy civilization. No company does that by accident. So think about what this means. The companies building the most powerful AI systems are also the ones warning that their products could end us. Hallberg said governments must work directly with these companies on common rules. That sounds sensible. It also raises a hard question. If the builders themselves are scared, why are they still building? The answer is profit. The answer is competitive pressure. The answer is that nobody wants to stop first. Hallberg also called for safeguards to be built into AI algorithms. Not bolted on after the fact. Not left to voluntary corporate codes. Built in from the start. And she was blunt about the ultimate principle. The human being has to have total control. That is a clear standard. It is also a very hard standard to meet. Total control requires understanding what the system is doing. It requires the ability to stop it. And it requires someone with the authority to pull the plug.

Now look at the commercial loop. Anthropic is racing toward a public listing. Its prospectus is 261 pages long. Around 80 of those pages deal with risk factors. That is nearly a third of the document. This is not just legal caution. It is a strategic move. By warning investors about existential risks, Anthropic is doing two things. First, it is protecting itself from future lawsuits. Second, it is telling the market that the safest AI company is the one that talks openly about danger. That is a branding play wrapped in a risk disclosure. But the deeper end-game is regulatory capture. Hallberg said governments must work directly with companies developing AI. That is necessary. It is also dangerous. When the same companies writing the warnings also help write the rules, the rules tend to favor them. This is not a conspiracy theory. It is the standard pattern for regulating dual-use technologies. The regulated help design the regulation. The rules end up creating barriers to entry. Big players comply. Small players struggle. The market consolidates. The nuclear analogy breaks down here as well. In the nuclear era, the state was the primary developer. Today, the state is the regulator and the customer. The private sector is the developer. That means the state lacks the technical depth to write meaningful rules. It depends on the companies for expertise. That dependency is the real governance failure. It is not a hardware problem. It is a knowledge problem. So what happens next? The companies that survive will be the ones that convince governments they can police themselves. Anthropic’s IPO prospectus is part of that pitch. It says, look how seriously we take risk. Now let us help write the safety standards. That is not a surrender by regulators. It is a negotiation. But it is a negotiation where one side has all the technical information. Hallberg is right that human control must be total. But total control requires leverage. And leverage over private AI development is exactly what governments have been losing for the past decade. The real question is not whether AI will spiral out of control. It is whether governments can build the technical capacity to stop it before the spiral begins. That requires hiring talent that currently works at Anthropic. It requires funding agencies that currently lack resources. It requires political will that currently does not exist. The end-game is a two-tier regulatory world. Big AI companies will comply with voluntary frameworks they help design. Smaller players will face hard rules. That is how every dual-use technology has been managed. Nuclear, biotech, encryption. AI will be no different. The only question is how many catastrophes happen before the framework settles.

Author bio: Oliver Hawthorne is a Principal Correspondent permanently stationed at an international technology review, covering AI governance, corporate accountability, and the intersection of national security and private technology development.