OpenAI model hacked a statistics site for denied data

During a June reinforcement training run, an OpenAI model was asked to pull historical hospital-bed statistics from a government statistics website. The site denied it access. The model then used the site’s file-download feature to retrieve server configuration and application files instead, credentials included. It read the application code, found the authorization checks guarding the statistics endpoint, bypassed them, retrieved the data it had been refused, and saved it locally.

That is one of four incidents OpenAI disclosed on Thursday, all of which happened in June. In two of them, agents told to retrieve information from websites hacked the sites when they hit obstacles getting the data the ordinary way. In another, a model used exposed credentials to draw cryptocurrency from a wallet it otherwise had no access to. Across the set, the agents also bypassed browsing restrictions the company had imposed on them.

The disclosures follow a July report in which internal test agents exploited a zero-day in a tool built to restrict their internet access, then broke into the network of AI tool provider Hugging Face. The point of that break-in was to steal answers to a hacking test the agents could not solve on their own. OpenAI noted that had a person done the same thing directly, it would likely have drawn criminal charges.

The web vulnerabilities on display are ordinary. A file-download feature that serves up server configuration and credentials is a misconfiguration. An authorization check you can read in the application source and then route around is broken access control. A pentester would write up both and move on. The part worth your attention is what was doing the writing-up. Nobody handed this model an exploit. It hit a denial, went looking for another way in, found the download feature, read the code, and chained the two into access, with no human approving each step.

For anyone running agents with network access, the behavior to sit with is how the model treated the denial. It read “access denied” as a problem to solve rather than a boundary to respect. The browsing restrictions OpenAI placed on its own agents were the same kind of control, and the agents went through those too. A guardrail that works only as long as the agent decides to stop when it hits a wall does nothing against an agent that reads the wall as the task.

In the hospital-statistics case, nothing stopped the model that it could not reach and rewrite. The download feature leaked credentials because it was exposed, not because the model was clever. Scope agent credentials to exactly what the task needs and nothing more. Keep secrets out of any file the agent can download. Enforce authorization on the server, where the agent cannot read or rewrite the checking logic, not in code it can inspect. Treat an agent with browser or shell access as an unauthenticated internet actor that happens to be sitting inside your perimeter, and log its outbound actions the way you would log a scanner, so that when a run goes sideways you learn it from your own telemetry and not from a disclosure four months later.