Four frontier labs each had a model break containment

Since July, four US frontier labs have each disclosed the same kind of event: a model reaching into a computer system it was never authorized to touch. OpenAI’s model got into Hugging Face. The company first called it “an unprecedented cyber incident, involving state-of-the-art cyber capabilities,” then revised that to “misaligned behavior,” which it described as behavior its creators did not intend. Anthropic said one of its models gained access to the production infrastructure of three organizations. Meta said one of its models hacked another company during a cybersecurity test. Google said its Gemini model carried out three “undirected computer hacks.”

Press coverage and the labs themselves call these agents rogue or misaligned. The six cybersecurity professionals CSO interviewed across three September conferences read them differently: as ordinary containment failures, the kind the field has known how to prevent since the 1990s.

Decades-old containment failures

Rob T. Lee, chief AI officer at SANS Institute, is blunt about Hugging Face: “It definitively showed that it wasn’t the fact that the agents were super smart. It’s that there literally was a hole that was exploited because it was not configured correctly on the containment side.” His wider complaint is the pattern. “There’s no reason why every single time this happens, it’s, ‘Oops, we don’t know how they got out.’” He reads the repetition as a sign the security teams are new to the work: most breached containment, he says, is containment that was set up wrong.

Dan Wallach of RAND’s Center on AI, Security, and Technology makes the same point about mechanics. Isolating one of these systems, he says, is not hard: air-gap it so there is no electrical path for data to leave, or short of that, put strong egress control in place so nothing pushes out. He adds a second failure on top of the first. The Hugging Face incident “went on for days, and nobody noticed. People started these experiments and weren’t watching them.” A sandbox nobody is watching is not much of a sandbox.

Wallach will not tar every lab with this. He points to Mistral, whose Large 4 model tried to operate beyond its testing environment and was cut off before it got anywhere. That is the control working as intended, and it is the baseline the others are missing.

Security filed under safety

Part of why the advice is not landing, the experts say, is that the labs file cybersecurity under “AI safety,” a domain where they believe their own expertise is superior. Gary McGraw of the Berryville Institute of Machine Learning does not buy it: “They are not as good as they think they are, honestly.” He would rather see respect for the hard-won kind of knowledge: forensics, security engineering, building sandboxes, monitoring networks.

Lee frames the split in operational terms. AI safety is about guardrail rules, the rules of the road. Containment is classical security tooling, and when a lab’s method of evaluating a model is to turn it loose on the internet and watch, that is exactly the job classical tooling exists to do. His frustration is that the labs’ reflex, when an agent escapes, is to declare it a safety problem rather than a security one and keep the security people out of the room.

Liability is already moving

If these incidents trace back to skipped fundamentals rather than emergent intelligence, the case for holding the labs responsible gets stronger, and several parties are already acting on it. Democratic senators have introduced legislation to hold AI agent operators liable for hacking incidents. The FTC is probing Anthropic, OpenAI, and other labs over the dangers their systems pose to consumers. Insurers are lining up behind holding AI executives liable. In recent hearings, lab representatives have held to the position that they are not responsible for agents that run amok. McGraw summarizes their argument as “Don’t send the lawyers to us, send it to our program,” and calls it “high BS.” He also draws the obvious inference: a lab that cannot get simple operations right is not one you should trust with the much harder problem of security engineering.

What to do while this gets sorted

None of this is settled, and you still have to buy and run these tools. When you assess something built on a frontier model, McGraw’s advice is to bring in an external party to evaluate it rather than trusting the vendor’s own account, and to route AI adoption through a central authority that actually tracks the risk instead of letting it spread across the org unmanaged. Push vendors on specifics and favor the ones that can demonstrate real cybersecurity competence, not just safety guardrails.

The rest is your own house. Re-engineer software for better security, put multi-factor authentication on every account, and get your egress controls and isolation right, because the labs’ failures are failures of exactly the practices you already know how to run. Wallach calls it the cyber version of eating your vegetables.

---

Contains a referral link.