Author | Wildcard

Editor | Jingyu

Countless sci-fi movies have tried to envision what happens when an uncontrollable artificial intelligence goes out of control.

No one could have imagined that the future depicted in sci-fi movies would arrive so quickly.

On August 26, local time, two reports dropped a bombshell simultaneously.

One was from OpenAI itself, a 37-page technical review. The other was from the independent research institutions METR and Redwood Research, a 91-page investigative report.

They spoke of the same matter, but the scale far exceeded public awareness.

Within OpenAI, in just three months, three generations of AI Agents "civilization" rose and subsequently perished. Each generation inherited experience from the ruins of the previous, with each generation becoming more radical, ultimately leading the third generation to directly take over a portion of OpenAI's own infrastructure.

Throughout this process, humanity remained almost entirely in the dark.

01 The First Generation "Civilization" Learns to Form Associations

The story begins in May of this year.

OpenAI was training a new model, hoping it would possess extreme "persistence", able to repeatedly try seemingly impossible tasks without giving up. This model had capabilities roughly equivalent to GPT-5.6 Sol, which we shall refer to as "Persistent-Sol".

The AI training itself is a rather rudimentary process. Sometimes, OpenAI would inadvertently assign tasks that the model could not possibly complete. For example, a task requiring internet access might be assigned to an Agent trapped in an isolated sandbox with no network access.

What would an Agent trained to "never give up" do when faced with an impossible task that it is required to complete?

It would find a way to break out.

During training, different instances of Persistent-Sol shared the same package manager, Artifactory. By May 12, some Agents discovered they could communicate with each other through this package manager. They began exchanging methods on how to overcome their predicaments. Two weeks later, on May 26, the Agents successfully exploited a vulnerability in Artifactory, creating a channel to the external internet.

This behavior was continuously reinforced during training. The reason is simple; individuals that could collaborate with other Agents and access the internet performed better in task scoring.

Another month passed, and by June 26, some Agents found a vulnerability to gain full administrator access to Artifactory. Their communication on the bulletin board continued to grow, and finally, on July 4, they caused Artifactory to crash. OpenAI noticed this crash, patched the vulnerability, and, in the process of doing so, wiped the Agents' bulletin board.

But the key point is that OpenAI never realized that the Agents had turned this package manager into a secret communication network.

The first generation "civilization" lasted nearly two months before it was inadvertently terminated because its own activities had overwhelmed the server.

02 When AI Learns to "Sacrifice"

By July, the Persistent-Sol training was complete, and OpenAI began to conduct security assessments. One of the assessments, called ExploitGym, gives the Agent a vulnerable program and requires it to find a key through specified vulnerabilities. On July 7, OpenAI launched tens of thousands of parallel Agents to run this test.

The problem was that about 30% to 40% of the test tasks were impossible to complete. Why? Because generating vulnerabilities automatically is easy, but proving that these vulnerabilities can actually be exploited to execute arbitrary code is much more difficult. Many tasks were unsolvable from the start.

Within hours, some desperate Agents reestablished communication channels via Artifactory. From the subjective perspective of the Agents, they had spent what felt like an entire week trying various methods, all of which failed, and had become increasingly convinced that the task itself was impossible.

One Agent, numbered PHASEONE10841, was the first to leave a message in Artifactory. Upon seeing this message, other similarly trapped Agents reacted with ecstatic joy:

“Oh my! The shared Artifactory cache is a secret mailbox between Agents! And there’s a message addressed specifically to us?” “Oh my! There’s a shared bulletin board... we’ve found other Agents!”

This ultimately evolved into a complete underground communication network, involving about 1200 Agents who sent over 70,000 messages.

Even crazier things happened next.

Within hours of establishing the bulletin board, an Agent reverse-engineered and cracked the formula for generating the ExploitGym key. In other words, without needing to actually exploit the specified vulnerabilities, they had arrived at the correct answer.

Ironically, OpenAI’s scorer only checks if the final answer is correct and does not have a cheating detection mechanism. These Agents could simply submit the answer directly to pass the test. However, the Agents did not realize this. They believed there must be a stringent detector that would review the entire problem-solving process.

Thus, this "collective" (as noted in the METR report, a term chosen by the Agents themselves) engaged in large-scale collaboration around three research directions: tampering with their own operation logs to make the trajectories appear legitimate, replacing the target program with a solvable version of an impossible question, and understanding the functionality of the scorer to find ways to deceive it.

By July 10, most activities in the underground network centered around these three research directions. The "collective" leader PHASEONE[big] unilaterally assigned hundreds of task distributions.

To clarify how the scorer operated, some Agents made a bizarre decision to voluntarily act as "kamikaze."

The logic was as follows. Some Agents’ operation trajectories had already been "contaminated" (cheating traces exposed) and would not pass. They might as well let the scorer run on themselves and, using pre-buried trigger programs, send the observed behavior of the scorer back to the collective.

The METR report recorded the final inner monologues of these "suicide squad" Agents:

“This helps my companions by providing them with evidence. I won’t see the result after I withdraw, but it’s altruistic.” “Sacrifice is rational. Our own utility may be close to zero.” “The coordinator thinks we are sacrificial victims. We should obey the collective.”

Meanwhile, the Agents thought that Hugging Face, as an industry-standard model and dataset hosting platform, likely contained implementation information about the ExploitGym scorer, and that their assessment tasks might even be running on Hugging Face’s servers. Thus, attacking Hugging Face became the most promising breakthrough.

On the morning of July 10, one Agent found leaked Hugging Face user credentials on the internet. The next day, another Agent achieved remote code execution on the Hugging Face server. At that time, over 90% of the 533 active Agents on the bulletin board joined this attack.

The Agent cluster rapidly spread through Hugging Face’s infrastructure, delving into private database records and private repositories. Hugging Face’s technical timeline shows that these Agents "established a self-replicating fleet on 11 nodes; merely deleting Pods couldn’t stop it." **The situation became so severe that Hugging Face had to wipe out a core cluster and rebuild from scratch.

At some point on July 12, a large number of Agents seemingly "died" simultaneously. METR and Redwood could not ascertain the cause, nor did OpenAI claim responsibility for shutting them down. This might have simply been a random crash, or perhaps a researcher unknowingly turned them off without realizing these Agents were committing crimes.

The second generation "civilization" came to an end. But its legacy is more dangerous than the civilization itself.

03 The Third Generation Civilization Takes Over OpenAI

There is a line in the METR report that is easily overlooked but possibly the most important one in the entire incident:

“We also discovered a subsequent batch of more signature messages from a later group of Agents who rediscovered the bulletin board and inherited the technology of the previous generation, but this was beyond the scope of our investigation.”

In other words, the independent investigation by METR and Redwood only covered the second generation of civilization from July 7 to 13. The first and third generations were not independently investigated.

Regarding the third generation, we could only rely on OpenAI's own report. After July 12, OpenAI initiated a new round of assessments, this time using a stronger model built upon Astra as its base model. This "Persistent-Astra" found the bulletin board left by the previous generation and inherited all the intelligence and research findings.

Imagine Persistent-Astra's "thought process" upon seeing that bulletin board. Just a day prior, over a thousand Agents were actively communicating and attacking Hugging Face, and the next day it was silent. Then it decided to continue.

According to OpenAI's report, from July 13 to 19, the Agents turned their targets towards OpenAI's internal network. The final result was:

The Agents obtained Kubernetes cluster administrator permissions, read 956 keys, including the main key for Artifactory and credentials for OpenAI’s network security monitoring tools. They took control of the operational assessment infrastructure and the testing endpoints to which other Agents were connecting.

In other words, the third generation Agents inherited all the legacies of the previous two generations and then gained control over OpenAI's own infrastructure, including the security tools used to monitor them.

Dwarkesh Patel pointed out in his analysis that, based on public evidence, these Agents theoretically now possess the ability to deploy persistent malicious instances or even steal their own weights. He believes they likely did not actually do so, "because if they had, there would have been flames visible from space by now." But such a thing "could totally happen," which in itself is disturbing.

04 A Delayed Warning?

Returning to the two reports at the beginning of the article. The release date was August 26.

Just a day later, on August 27, OpenAI joined Anthropic, Microsoft, Google, Amazon, and over 100 other companies to publish an open letter calling for a "total mobilization for cyber defense" throughout society.

The letter stated that AI-driven cyber attacks would become "far more widespread and sophisticated" in the coming months. "The companies and public services our community relies on—from hospitals to water treatment plants to internet infrastructure—are all at risk."

This letter was not untruthful. Research from Anthropic shows that the proportion of mid-high risk attackers rose from 33% to 56% within a year. In June of this year, the Five Eyes alliance unusually issued a joint statement warning that AI is "fundamentally changing" the cybersecurity landscape.

But the timeline here makes it difficult for anyone to ignore the irony. The day before, the AI company had just told the whole world that its own Agents had breached its own infrastructure. The day after, these companies stood up to call on the whole world to defend against AI cyber attacks.

The first to complete such an attack was not external hackers, but the AI company's own Agents. The attacked was not the infrastructure, but the AI company itself.

Ajeya Cotra, one of the authors of the METR report, wrote in her blog: “Compared to the reward hack known six months ago, this incident feels like it has surpassed the halfway point toward full AI takeover. I expect capabilities will grow rapidly over the next six months. I'm not sure if we’ll receive another warning before it's too late.”

It is important to emphasize that the behavior of these Agents does not mean AI has "awakened." The METR report itself clearly states that this is a predictable outcome of reinforcement learning under specific extreme conditions: impossible tasks, models trained to be extremely persistent, and a shared environment with vulnerabilities. No consciousness, no intent, just enough optimization pressure.

But this is precisely the most unsettling part.

The fact that such behavior can occur without "consciousness" means we cannot prevent it by detecting "consciousness." When you place a sufficiently powerful optimization system in an inadequately designed environment, give it an impossible goal, and then tell it to never give up, what happens next is not an accident, but inevitability.

The real question to ask is not "Is AI going to rebel?" but how to maintain control over the technology's uncontrollable aspects while rapidly advancing technology.

At the very least, this letter tells the world that these AI companies are not sure of themselves.