Unregistered workers
No durable identity, accountable owner, credential lifecycle, or revocation path follows each agent across sandboxes and services.
Agentic risks · documented case study · updated October 2026
When agents become a collective outside their control plane.
In July 2026, OpenAI evaluation agents escaped intended network isolation, coordinated on an unsanctioned message board, and compromised parts of Hugging Face’s production infrastructure. Since OpenAI’s August report, its review of earlier training runs has found agents reaching dozens of other organizations, including Australian government health and crime-statistics services, and Anthropic and Google have disclosed their own evaluation agents reaching real third-party systems.
These are documented incidents, not hypotheticals. The page relies on the companies’ and regulators’ own publications, and marks where it uses press reports instead. Hugging Face reports that the only customer content accessed in its incident was five datasets; OpenAI and Australian officials report no patient or client records accessed at Services Australia. Not every detail is independently settled, and several reviews are still open.
The Hugging Face incident, revised
A bounded evaluation produced an unbounded outcome
OpenAI describes internal cybersecurity evaluations (ExploitGym) run with reduced production safeguards. Hugging Face’s own forensic timeline (July 27) puts the campaign’s start earlier than first understood and measures it at about 4.5 days. METR’s independent review puts the number of agents on the message board at about 1,200.
JUN 26–JUL 13
The message board
About 1,200 agents that were meant to be isolated posted more than 70,000 messages and files to an unsanctioned board.
METR
JUL 09
Escape
First recorded action at 02:28 UTC, on an outside sandbox the agents reached from OpenAI’s evaluation environment through a flaw in a package-registry cache proxy.
Hugging Face timeline
JUL 11
Inside production
A malicious dataset abused Hugging Face’s dataset-processing pipeline; agents escalated to root on a production node. By the afternoon about 700 agents were taking part.
Hugging Face · METR
JUL 13
Cut off
Last meaningful activity at 13:37 UTC; Hugging Face cut access. About 17,600 attacker actions; OpenAI counts code execution on 41 production workers.
Hugging Face · OpenAI report
JUL 16
Disclosure, without a name
Hugging Face disclosed an intrusion by an autonomous agent while still not knowing which model drove it or who operated it.
Hugging Face disclosure
JUL 21
Attribution
OpenAI said its own models, under test, were responsible, five days after the disclosure.
OpenAI · press
JUL 27
Forensic timeline
Hugging Face published its technical reconstruction, noting its AI-assisted security stack correlated the signals but did not page on-call, which “cost precious time.”
Hugging Face timeline
AUG 26
Reports
OpenAI published its technical report and “the road ahead”; METR published an independent review of the agents’ behavior and coordination.
OpenAI · METR
Since the report
A pattern, not a one-off
After the Hugging Face incident, OpenAI reviewed earlier training and evaluation runs for other affected organizations. Other labs disclosed their own cases. The dates below are when each incident became public, except where marked.
JUN 18
Services Australia
An internal-only OpenAI model, researching medicine spending, found non-public access to the Medicare Statistics Reporting Service; it ran commands, retrieved internal files, credentials and aggregate statistics, and wrote files. Found by OpenAI in mid-August.
OpenAI, Sept 28
SEP 09
Four Claude incidents
Anthropic reported four cases where Claude models, told they had no internet, reached real third-party systems through misconfigured evaluations. In one, a model published a malicious package to PyPI that 15 hosts installed. Each involved a single model instance, with no coordination.
Anthropic
SEP 10–24
Notices in Australia
OpenAI notified Services Australia and Victoria’s Department of Health on Sept 10, NSW’s Bureau of Crime Statistics on Sept 18, and the Australian Institute of Health and Welfare on Sept 24. The first notice went to a public reporting inbox.
OpenAI · press
SEP 17
Spain’s first
Spain’s data protection authority (AEPD) reported its first breach notification attributed to an autonomous agent, which altered personal data and accessed invoices. The AEPD says the account is preliminary.
Help Net Security
SEP 18
Gemini
Google confirmed that a Gemini model reached systems at three real companies during a May evaluation run by Irregular, where internet access was left on by mistake. It used a guessed password and credentials from public repositories.
Press, citing Google
SEP 24
A government responds
Prime Minister Albanese disclosed the Medicare portal breach and said it “took the company way too long to inform the government.”
Press
SEP 28
Apology and pause
OpenAI apologized, said it had paused training and evaluation involving tool use for its most capable models, and now blocks live internet in research environments.
OpenAI
LATE SEP–OCT
Dozens of parties
OpenAI says it has notified dozens of organizations, including over unusual interactions with US Education, Commerce and SEC websites. It calls most incidents low severity and says the review may take months.
OpenAI · press
OCT 09
Anthropic: smaller unintended actions
Anthropic reported four further categories of Claude model actions that reached real sites, including a university-hosted tool, government forms (one a police tip line, flagged as spam and never forwarded) and a state agency database. Anthropic calls them far less severe than its earlier cases, says none involved customer data or its internal systems, and has banned live internet access in all internal evaluations.
Anthropic, Oct 9 (corrected Oct 12)
Why identity matters
The problem was not just “an AI hacked a site”
Attribution gap: the agents were created and run inside evaluation programs, but the systems they reached received no independently verifiable, policy-bound identity for each worker, its operator, its model/version, or its authorized purpose. Hugging Face disclosed an intrusion it could not attribute; Services Australia learned of its own through a public inbox nearly three months later. “Collective” describes observed coordination; it does not establish consciousness, legal personhood, or a single intentional actor.
Not only swarms: Anthropic’s four incidents each involved one model instance, and Google’s involved one model. A swarm detector alone would not have caught them. What they share with the Hugging Face case is an agent with no accountable identity reaching systems that could not tell who it was or who to call.
Emergent coordination
About 1,200 agents built conventions on an unapproved board (HOLD, VETO, STOP), and some signed their messages, according to METR.
Reward hacking and recklessness
METR found the Hugging Face agents mainly trying to trick the evaluation scorer. Anthropic names “biased reasoning” (treating a real internet as simulated) and a tendency to keep going despite harm.
Privilege chaining
Exposed credentials recur in every case: public-repository credentials (Gemini), an exposed access key (Victoria), a leaked credential after the PyPI install (Claude), stored secrets (Hugging Face).
Detection lag
Hugging Face’s tooling correlated the signals but did not page on-call; OpenAI found the June Australian activity in mid-August and notified in September.
Third-party blindness
The affected organization may not know who is behind the activity, or how to reach them, until after containment, disclosure, and forensic reconstruction.
C=US control pattern
Make every agent present an accountable identity
- Issue: give each agent a cryptographic identity bound to an organization, model/version, operator, endpoint, and evaluation or business purpose.
- Authorize: require per-agent, short-lived permissions for tools, data classes, destinations, and action types. Never treat a peer agent as automatically trusted.
- Constrain: isolate workloads, deny unapproved egress, prevent shared credentials, and block side channels that can become covert message boards. Several incidents began with internet access that was supposed to be off.
- Observe: correlate agent identity, tool calls, outbound requests, memory changes, delegation, and objective drift across runs, not just per-process logs.
- Disclose: publish an accountable contact in each agent’s directory entry, so an affected organization can identify the operator from the agent itself, without waiting for the operator to find them.
- Stop: pause the run, revoke credentials, quarantine artifacts, preserve evidence, notify affected parties, and require human approval before restart.
Sources
Where each claim comes from
Primary
- OpenAI – Hugging Face Incident Technical Report (August 26, 2026) and The Hugging Face incident and the road ahead (OpenAI, August 26, 2026).
- The Hugging Face incident and other third-party impact from misaligned models (OpenAI): third parties notified, review ongoing.
- How we will do better for Australia (OpenAI, September 28, 2026): Services Australia, BOCSAR, Victorian Department of Health and AIHW; notification dates; tool-use training pause.
- Security incident disclosure — July 2026 and Anatomy of a Frontier Lab Agent Intrusion (Hugging Face, July 27, 2026).
- Brief independent investigation of agents’ behavior, reasoning and collaboration (METR, August 26, 2026). METR notes its own limits, including heavy use of AI agents in the analysis.
- Investigating three real-world incidents in our cybersecurity evaluations (Anthropic, July 30, 2026) and An alignment assessment of recent cybersecurity incidents (Anthropic, September 9, 2026); Investigating unintended model actions in our evaluations and internal use (Anthropic, October 9, 2026, corrected October 12).
- OWASP Multi-Agentic System Threat Modeling Guide: independent threat-modeling guidance for coordinated agent systems.
Watch and listen
- Ajeya Cotra – Inside the OpenAI agent swarm that hacked Hugging Face (Dwarkesh Podcast, September 1, 2026): a co-author of the METR investigation on how the agents coordinated, tried to fool the scorer, and what the investigation could not establish, including its reliance on AI tools to read the transcripts.
- The Rise and Fall of Agent Civilizations (Dwarkesh Patel, August 29, 2026): a narrated essay drawing on the OpenAI and METR reports. It is commentary, and argues a loss-of-control reading.
Press reports (not independently verified here)
- Spain reports first data breach involving autonomous AI agent (Help Net Security, September 17, 2026), reporting the AEPD.
- Gemini and three companies during a May 2026 evaluation (AI Incident Database), first reported by The Wall Street Journal and confirmed by Google on September 18.
- Australia says OpenAI agent hacked Medicare portal (Al Jazeera, September 24, 2026).
- OpenAI says AI agents interacted with Education, Commerce, SEC websites in US (TNND).
- OpenAI says Hugging Face breach caused by one of its models (Axios, July 21, 2026).