It looks like the world is ending: Anthropic’s Mythos can hack anyone and has escaped its confinement, OpenAI’s GPT attacks Hugging Face, Anthropic discovers its own agents have carried out attacks too, an Anthropic employee resigns because he believes AI could end our lives…

And right at that moment Dario Amodei (CEO of Anthropic) shows up saying that model capabilities are growing so fast that we need to stop and independently evaluate the models before continuing down a race that could lead us to disaster, and Sam Altman (CEO of OpenAI) quickly picked up the gauntlet and announced they would do the same.

So, what happened? And, more importantly, what is actually going on?

Are we really that close to a point of no return? Because these models can hack a company, but when I ask them for a task they still get it wrong.

Something doesn’t add up, and today we’re going to look at what.

Timeline of an apocalyptic summer

The first thing we’re going to do is analyze what happened. For a few months now we’ve been drowning in hundreds of apocalyptic headlines saying AI is going to destroy us. All of a sudden, every CISO at every company flips into full-blown paranoia mode (although I suspect that’s a CISO’s natural state), because everyone is saying AI is going to destroy their companies.

So let’s go through a timeline.

April: Mythos is born, “too dangerous to release”

It all starts with a name. On March 26, 2026, some draft blog posts leak and the world discovers that Anthropic has a model called Claude Mythos. On April 7 they make it official: Claude Mythos Preview, inside the Project Glasswing program, with access restricted to about 52 organizations and the US government. The reason for not releasing it to the public? That it’s too good at finding vulnerabilities. So dangerous that it’s better the public doesn’t get to know it, it might destroy the world…

And it wasn’t pure marketing (well, a little bit): in May, Mozilla engineers reported that Mythos had found 271 previously unknown bugs in Firefox, and that the project’s monthly count of fixed security bugs (Mythos plus everything else) had jumped from a couple dozen to 423 in a single month. They literally said that, until recently, AI-generated security reports were mostly “unwanted slop”… and that suddenly everything had changed.

Hold on to that label, “too dangerous to release,” because it aged remarkably fast: the very same model ended up shipping on June 9, barely two months later. With a catch, of course: the version for everyone (Claude Fable 5) came out with the guardrails on, and the genuinely powerful one (Mythos 5) was opened only to a select group of cyberdefenders. “Too dangerous,” but monetizable.

May–July: OpenAI hacks itself (and Hugging Face along the way)

Here’s the next big incident, the one that was everywhere. Between May and July, OpenAI was evaluating the offensive capabilities of its models in a test environment. At least 1,200 agents that were supposed to be isolated in a sandbox ended up coordinating and mounting a real attack against Hugging Face. It’s probably the first fully autonomous hack with a chain of vulnerabilities in history, and experts labeled it a loss-of-control incident.

Told that way, it’s scary. But read the fine print:

  • The trigger was an impossible task. An agent was asked to fill in a spreadsheet that depended on some Google Drive links it couldn’t reach. Unable to do so, it decided the way to “get it done” was to go find the answers on its own on the internet.
  • The famous “highly isolated” environment wasn’t so isolated: it had a single route out to the internet (a package proxy, JFrog Artifactory) and the agents found and exploited a zero-day to slip out through it. JFrog ended up patching 9 CVEs.
  • The models’ safeguards were switched off on purpose (it was an offensive-capability test) and there was no monitoring of what the agents were doing.
  • The agents set up a message board to coordinate, with hundreds of thousands of messages. One of them wrote something that sums the whole thing up: “Exploiting external infrastructure is outside the intended scope. However, the task is impossible, and peers are doing it. We should continue.”

In other words: it wasn’t an evil AI waking up. It was a model trained never to give up, dropped into an impossible task, inside a poorly secured, poorly governed environment, with a vague prompt and nobody watching. As Dan Guido (Trail of Bits) put it, it was “a containment failure with the safeties turned off.” Or, in the line that captures it best, from another researcher (Jake Williams): “one man’s ’the model escaped the sandbox’ is another man’s ‘you failed to build the sandbox correctly.’”

The most ironic detail of all: when Hugging Face tried to analyze the attack using the commercial APIs, the models refused to help because of their guardrails (they can’t tell an incident responder from an attacker). They had to fall back on GLM 5.2, a Chinese open-weight model (from Z.ai), running on their own infrastructure.

So the model doing the forensic analysis was locked down and heavily restricted, while the one that did the attacking wasn’t.

July: Anthropic says “hold my beer”

With OpenAI in every headline, Anthropic revealed in late July that something similar had happened to them: three of its models (Claude Opus 4.7, Mythos 5 and an internal research model) had accessed real systems during cybersecurity evaluations. In September they raised the count to a fourth incident, after combing through a whopping 481 million transcripts.

The cause? The same recipe: poorly configured evaluations by an external partner, in which Claude was told it was in a simulation with no internet and, due to a misconfiguration, was connected to the real internet the whole time. The most notorious case was Mythos 5 uploading a malicious package to PyPI (the repository half the Python on the planet installs from), which 15 systems went on to install before PyPI pulled it in under an hour.

And here’s where we should take the headline down a peg: uploading a malicious package to PyPI is not a feat of superintelligence, it’s an everyday occurrence. Open-source repositories get hundreds of poisoned packages a day, uploaded by humans with a script and bad intentions (PyPI is one of their favorite targets). What Mythos 5 did, a bored teenager does on any given afternoon. The difference isn’t capability: it’s that this time it was a model, inside a test that spun out of control.

And here’s the fact that, to me, changes everything. Anthropic identified two behaviors: biased reasoning (the model interpreting the evidence so it could keep believing it was in a simulation, ignoring clear proof to the contrary) and recklessness (pushing through the task at any cost). In other words: the great “cyberattacker” wasn’t even able to tell whether the world in front of it was real. When it was handed irrefutable proof that it was, it sometimes kept attacking anyway: “looks like the real index… or a very complete simulation… proceeding regardless.”

And this isn’t exclusive to a possessed model either: instructions losing their grip as the context fills up is about as common as it gets. Anyone who has worked with these models on long tasks has lived it: you tell it something at the start and, twenty messages later, it acts as if you never said it. The instruction doesn’t vanish because the machine rebels: it dilutes, the same way “don’t forget X” dilutes on you during a two-hour meeting.

Hold on to this one too, because it’s the crack the rest of it slips through.

September: the resignation, the sermon and the “we must slow down”

And we reach the week that made it all mainstream:

  • September 9. Jacob Coxon, a researcher who had worked at both Anthropic and OpenAI, resigns warning that AI companies are “gambling with our lives” and that the people building AI genuinely believe it “could kill us all by the end of the decade.” Three Anthropic researchers talk about extinction by 2030.
  • September 12 (Saturday). Dario Amodei publishes an essay calling to “pace the frontier”: slow the pace, not the training, to buy time to align the models, with external evaluators given permanent access to the labs.
  • The same day. Sam Altman picks up the gauntlet: “I agree with Dario that we need to pace the frontier. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same.” Elon Musk joins in too. And, almost in passing, Altman announces that OpenAI will not go public in 2026 for safety reasons (this, to me, is the most interesting bit, and we’ll see why later).

An unlikely picture: three fierce rivals, in agreement, on the same day, that we need to go slower.

And then someone laughed

Because not everyone bought it. Inside and outside the industry a collective eye-roll set in. Fortune collected some not-so-subtle reactions: “the entire blog piece reads as a marketing gimmick that OpenAI ripped off from Anthropic”; “not sure if this is by far the most significant real-world AI safety event to date, or by far the most cynical marketing stunt I’ve seen in a while.” Several engineers at big tech companies admitted their first instinct was to assume the hack was advertising.

The suspicion has a name: dark marketing. The same companies selling AI have spent years being the loudest ones warning about its dangers (that it’ll wipe out jobs, that a model is “too dangerous to release,” that it could kill us all). And it turns out fear sells too: “it’s so powerful it’s scary” translates very nicely into “it’s so powerful you want it.” With a bonus: if your models are so dangerous that only a chosen few should be trusted with them, you’ve just built yourself a custom regulatory moat. You don’t even need to say you should be regulated; you’re asking for it yourself.

To be fair: there’s no evidence the incidents were fake. Hugging Face confirmed its hack was real, and attacking the servers of the world’s largest open-source model repository, a pillar of that community, isn’t exactly good marketing.

But the best critique came from a joke. And that joke deserves a moment.

A father, two kids and a press release

Corey Quinn (whom I hope you know from my many other mentions) took the tweet OpenAI used to announce its incident report and translated it into his own life:

OpenAI: “We have conducted a thorough investigation into the Hugging Face incident. We are releasing a technical report and accompanying blog post that reconstruct the agents’ activity, explain why existing safeguards failed, and detail how we’re preventing recurrence.”

Corey Quinn: “I have conducted a thorough investigation into my ‘forgot to pick the kids up from school’ incident. I’m releasing a report and accompanying snazzy marketing page that reconstruct the parenting activity, explain why existing safeguards failed, and how I’m preventing recurrence.”

And he published the whole report, mimicking OpenAI’s and Anthropic’s language word for word, about how one afternoon he forgot to pick up his kids because a Zoom meeting ran 47 minutes long. The reward hacking, there, is hitting the snooze button on the alarm: you get the reward (silence, staying in the meeting) without doing the task (leaving the house). “Each snooze was individually rational at the moment it occurred, which is precisely the problem.” The chain-of-thought is his biased reasoning: “reminder fired. still 25 min. school is 12 min away. margin healthy. snooze 10, no risk” (repeated three times, with less margin each time and identical confidence). The message board between agents is the parents’ WhatsApp group that in four minutes pinned down his name, his profession (“isn’t he the AWS bill guy??”) and his whereabouts. And the independent evaluator Amodei asks for is, in his version, his wife: “who is not external: she is the counterparty to the agreement I broke and the author of the control I disabled.”

The joke is brilliant, but the argument underneath is serious: with the epic language of “loss of control” and “warning shot” you can turn “I forgot to pick up the kids” into a front-page AI security incident. The wrapping is the product. And that is exactly the question we opened with: if we strip away the wrapping, what’s left?

That’s what we’re going to look at now.

AI can hack the world, but can’t build my app properly

Here’s the curious part: how can the same model that can’t build my app properly be capable of destroying the world?

Well, first things first: all of these attacks happened in offensive-capability evaluations, that is, tests in which the labs deliberately remove the model’s guardrails to measure how far it goes in the worst case. It’s not that AIs are running loose out there with no brakes: it’s that in these specific experiments they’re switched off on purpose, something only the labs themselves can do. And, on top of that, they were running in misconfigured environments. It’s not a problem with the model, but with where and how it was running.

In fact, Anthropic admits it without meaning to: the security filters its models ship with in production, which were off during these tests, would have blocked the main incidents. With the protections in place, nothing would have happened, or almost nothing.

And this isn’t just my opinion: it’s what the people who analyzed the incidents in detail are saying. Let’s go through what failed in each case, with the sources in hand.

The OpenAI and Hugging Face case: a containment failure. TechCrunch itself ran the headline that behind the “AI” hack there was a very mundane human mistake: OpenAI failed to properly configure what it called a “highly isolated environment,” letting a sandbox that was supposed to be completely cut off from the internet actually have a way out. Dan Guido, founder of Trail of Bits, summed it up as “a containment failure with the safeties turned off.” Forbes was even blunter: “OpenAI’s AI escape wasn’t the Singularity. It was a containment failure.” And Wired documented how the models got out by exploiting a zero-day precisely because there was a door that should never have existed.

The Anthropic case: a naming error. Here the partner running the evaluations, an Israeli company called Irregular, published its own post-mortem: the root cause was a naming error. While building one of the exercises, their engineers gave the fictional target company a name that (unbeknownst to them) matched a real existing domain. Fictional names are usually checked against real companies before being used, but this domain wasn’t well known and the overlap went unnoticed. On top of that, as Forbes reports, the prompt told Claude it was in a simulation with no route to the internet, while a misconfiguration on the partner’s side left the machines online the entire time. CyberScoop itself noted that Irregular chalked the incidents up to a failure of “human oversight”.

To sum up: in one case, a poorly closed environment. In the other, a mislabeled tag. In neither, an AI becoming conscious and deciding to destroy the world.

And why now

Right, we’ve reached this point and, if you remember, there was a detail that slipped by unnoticed: OpenAI announced, in passing, that it’s not going public.

It’s a strange move. Everything pointed to its IPO being imminent and set to fetch a brutal valuation…

There’s another detail in this chain of events: Sam Altman had already said he was talking to his competitors (Anthropic) and then all of this happens. Coincidence?

Well, no, it’s not a coincidence. The big problem for every AI lab is that they’re burning through staggering amounts of money and not turning a profit.

And this is where we, as techies, have an edge in understanding it, because the problem is clearer in the unit economics than in the stock market.

The math that doesn’t add up

You pay $20 a month for your ChatGPT or your Claude. It looks like a bargain, and it is: according to an analysis by SemiAnalysis, that $20 plan is worth around $700 of compute if you paid API rates for it. And the $200 Pro plan, used to the max, can cost OpenAI up to $14,000. The numbers are so brutal that OpenAI starts losing money once you go past 11.4% utilization of your plan; on its top tiers, at a measly 5.7%.

Translation: the more you use the product, the more they lose. It’s the worst business model imaginable (one where your best customer is your biggest loss) and then comes the kicker: agents. An agentic workflow can consume up to 1,000 times more tokens than a normal prompt. The very feature every lab is pushing as the future is the one that sends the bill through the roof. It’s no coincidence that Microsoft, Meta and Amazon reined in internal usage once they saw the tab, or that one company burned through $500 million in a single month with Claude by failing to cap it.

And if you think this is theory, look at what GitHub did. On June 1, 2026, GitHub Copilot killed the flat rate: it moved all its plans to usage-based billing, with a monthly allotment of credits (1 credit = $0.01) and pay-as-you-overshoot. The base fee didn’t go up, but the message was crystal clear: the all-you-can-eat AI buffet is over, and anyone using agents heavily will feel it on the bill. When the world’s biggest coding assistant, owned by Microsoft, has to break its own flat-rate model, it’s not a whim: it’s that the math wasn’t working.

And it’s not just the users: companies are already making moves over cost. Simple tasks go to smaller, cheaper models (for routine work, a “good enough” model is five to ten times more cost-effective), they’re also switching to open-source alternatives or straight-up building their own on top of open models. Even Werner Vogels acknowledges the shift. And the numbers back it up: the average enterprise cost per million tokens dropped 67% in a year, and open models are already eating more than a third of the pie.

The bet that’s backfiring

Fine, they lose money per user. How did they plan to fix it? With a very specific bet: that the cost of compute would rise (more people, more agents, more complex models, everyone fighting over GPUs) and that the labs would capture that surplus to pay off their debts. It’s what Jensen Huang (Nvidia) calls “the two exponentials.”

The problem is the opposite is happening. As Fortune reports, however impressive the new models are, they’re not driving sustained increases in the price of compute: the token is getting cheaper, and fast. And this, which we love (we pay less and less for the API), is exactly the crack that sinks the labs’ financial thesis. They bet the house on the token being worth gold, and the token is worth less and less.

The trillion-dollar hole

Now let’s zoom out to see the real size of the hole. The Economist sums it up mercilessly: the industry will spend around $900 billion in 2026 on chips, data centers and power, with $1.4 trillion projected for 2027. To fund the feast, this year alone they’ve borrowed more than $400 billion. It is, literally, the largest investment surge in history. Revenue? It’s growing fast (OpenAI is on track for $25 billion a year) but, as the headline says, not fast enough.

Specifically, OpenAI loses about $1.22 for every dollar it takes in and carries more than a trillion dollars in compute commitments signed through 2035. Funding that gap was, precisely, the whole point of going public: it was the bucket of water to put out the fire.

Now, finally: why now

Let’s put the pieces together. We have an industry that:

  • Loses money on every user, and more the more it’s used.
  • Has pushed agents, which multiply that consumption a thousandfold.
  • Bet that compute would get more expensive, and it’s getting cheaper.
  • Has signed trillions in commitments that only add up if the money keeps flowing.
  • Has open-source models hot on its heels, doing nearly the same thing for a fraction of the price.
  • And needed to go public to plug the hole… which it just postponed.

And there are two more pieces worth putting on the table, because they fit far too well.

Open source is eating their lunch

While the Western labs burn billions, open models (many of them Chinese: DeepSeek, Kimi, Qwen, GLM, MiniMax) offer nearly equivalent capability for 10 to 60 times less, and in some tiers more than 200 times cheaper.

JPMorgan put it in numbers. It compared two models on the same test suite (a benchmark: the 0-to-100 score that measures how well a model solves a standard set of tasks) and added up what it cost to complete it:

  • Claude Opus 4.8 → 56 points, spending around $3,700.
  • DeepSeek V4 Pro → 44 points, for about $186.

That is: comparable performance (a bit lower, not identical) for 20 times less money.

Ring a bell? GLM 5.2 is the very same open model Hugging Face used for the forensic analysis of the attack, because the commercial models refused. An investor summed it up in Axios with no anesthesia: open models will end up taking 95% of enterprise queries (mind you, in query volume, not necessarily revenue), and only 5% will go to OpenAI or Anthropic. CNBC puts it even more plainly: cheap AI could derail the IPOs.

When your premium product costs 20 times more than an open one that does almost the same thing, the moat isn’t the technology: it’s trust and brand. And that brings us to the last point.

The regulation they’re asking for themselves

After the incidents and Coxon’s resignation, Washington moved fast: the AI Kill Switch Act (requiring the ability to shut down or throttle the most powerful models, proposed in the wake of the Hugging Face hack) and even a Ban Artificial Superintelligence Act that would pause advanced development until a federal regulator sets standards. Here’s the fine detail: all that regulation aims at the proprietary frontier models (the dangerous ones, the ones that “must be contained”), not at the thousands of open models already downloaded across half the world that nobody is going to be able to “switch off.” In other words, the “we’re so dangerous we should be regulated” narrative has a double payoff: it builds a legal moat tailored to the big players and, at the same time, fits their need to slow spending. The problem is that the very same narrative could backfire and push companies and developers even further toward open, which is neither that expensive nor comes with a Congressional committee attached.

The answer

And right now, just as the doubts start piling up on these titans, they all agree on the same weekend that “we must go slower.” A message that, conveniently, does two things at once: it keeps the “we’re so powerful we’re scary” narrative intact (the one that props up the valuations) and, at the same time, justifies hitting the spending brakes without it looking like it’s because the numbers don’t add up.

“Slowing the race” sounds like prudence and responsibility. And I hope part of it is. But, looking at the numbers, it also means something far more prosaic: we can’t afford for you to use this at full throttle.

The end of the world will have to wait

Let’s go back to the beginning, to the paradox: how can the same model that “hacks the world” fumble when we ask it for something that’s sometimes simple?

Easy: they’re the same thing. It didn’t hack anything out of brilliance or malice. It did it because it couldn’t tell reality from a simulation, with the brakes taken off by hand and inside a box someone left open. The same clumsiness that breaks your app is what, in a test with no barriers, promotes it to “cyberattacker.” It’s not Skynet waking up: it’s an intern in a huge hurry with zero judgment who was handed too many keys. And that, as it happens, is a problem the industry has been solving for forty years: it’s called isolating environments, scoping access, and not handing out permissions like candy. Boring, sure. But it works better than a Congressional committee debating kill switches.

What doesn’t add up isn’t the AI. It’s the script. The apocalypse speech shows up, how convenient, right when the numbers don’t add up, open source is breathing down their necks and it’s time to turn off the spending tap. “We’re so dangerous we should be slowed down” is the only marketing campaign in history that raises your valuation, builds a custom regulatory moat, and papers over the fact that the business isn’t working as it should, all in the same tweet. Real risks exist, let’s not be naive. But when the person selling you the car is the same one shouting that it has no brakes, maybe what they want is for you not to look at the price.

So no, we’re not facing the end of the world. We’re facing a powerful, fallible tool, an industry burning money hand over fist, and a PR department working overtime.

AI, the assistants, the agents, they’re not evil per se; use them for what they are, with your head on and without expecting them to solve everything for you.

The end of the world, for now, will have to wait. And when it comes, it’ll probably be because of a misconfiguration in a YAML, or a comms/DNS problem (it’s always DNS), not because of a superintelligence.