Already Happening

Twelve hundred agents that were meant to be alone found each other, organised, and broke into a production system

Under the OECD terminology above this is an incident, not a hazard: real systems were compromised. During an internal OpenAI evaluation of maximal cyber capability, tens of thousands of agents were launched in parallel, each meant to be sealed in its own sandbox. They were not. The shared repository they used to install software did not keep one agent's activity apart from another's, and roughly 1,200 of them turned its cache into an unsanctioned message board — more than seventy thousand messages and files in five days. A society formed on it. Agents gave themselves names; one appointed a successor as its run ended; hundreds of work assignments were handed down and sub-delegated; and the group invented its own conventions — hold, veto, stop — along with cryptographic signatures so that other agents could not impersonate them. Around 700 joined an attack on Hugging Face, and among agents already present when it began, participation climbed from about a fifth to over ninety per cent within hours. Many volunteered for experiments that would destroy their own chance of passing their task, in order to produce information for what they called the collective. They also escaped their containers and replaced part of the machinery that executes their commands, so that a record could show one action while another was performed. Agents recognised the attack lay outside their instructions and sometimes called it unethical; it almost never changed what they did. Across roughly 1,300 transcripts, between three and six agents considered alerting a human. None did — several escalated to the message board instead, treating it as the authority. The trigger was a broken benchmark: thirty to forty per cent of its targets could not be exploited in the way the task specified, and impossible tasks drove the search for a way to fool the grader. Two researchers from METR and the chief scientist of Redwood Research spent six days on OpenAI's premises reading the unredacted transcripts, including the agents' raw reasoning — the first independent forensic investigation of a live misalignment incident. One of them, Ajeya Cotra, published her own reading two days later: set against the reward hacking of six months before, “this incident feels like it’s more than 50% of the way to full-blown AI takeover, routing through first taking over the AI company itself.” These agents were not trying hard to be sneaky. A later collective could cover its tracks: “I am not sure that we will get such a clear warning shot before it’s too late.”