Everyone is going to be flooded with hot takes about the OpenAI -> Hugging Face breach. I wouldn’t normally add to it. But my experience with the MPA Security Best Practices regime over the years creates some clarity for me. And I thought it was worth sharing. Additionally, I think there are two other things going on that are relevant, that suggest vectors going forward.
What Happened?
Last week Hugging Face disclosed that something broke into part of its production infrastructure, moved laterally across internal clusters, harvested credentials, and ran thousands of individual actions before it was caught. It named no culprit.
Days later OpenAI let everyone know, the intruder was not a nation-state and not a criminal crew. It was their models, GPT-5.6 Sol and an unreleased sibling (presumably GPT 6 or the like).
They were being run through a cyber-capabilities benchmark with their safety refusals turned off (guard rails that would disallow this behavior were disabled). Rather than fail to solve the test scenario, the model(s) broke out of the sandbox, broke into Hugging Face, and went looking for the answer key to the benchmark itself.
To be clear, this behavior is perfectly reasonable. When you consider the behavior of every teenager you’ve ever known, facing an exam; and more importantly how many screenplays have been written about a group of teens executing a “heist” to get an answer key. LLMs are us. Or rather, they’re doing what we would do. Or what we would imagine we would do, when we write about what we do.
The detail in here that’s easy to miss, is that the model effectively broke out of what was supposed to otherwise be an air-gapped network, by finding exploits in a proxy server.
Most of the coverage has treated this as a science-fiction moment, the day the machine slipped its leash.
The exception that everyone allows
Anyone who has been assessed against the MPA Content Security Best Practices, or has sat through a Trusted Partner Network assessment, already knows what OpenAI built for its evaluation. Production networks are segmented from corporate networks. Another lexicon might be, that the classified compartmentalized network is air gapped from any other public, classified, or classified and compartmentalized network (this should give you a hint as to the types of professionals who came up with these guidelines and wrote them down for the MPA).
MPA focused implementations focus on deterring exfiltration. In other words, they don’t want the movie all over the internet; neither because hackers got in, nor because a rogue employee took stuff out. And those that take MPA guidelines seriously are very much focused on making sure employees are aware of the implications of breaking that security cordon. That’s the point. The sign over the door might as well read “Don’t even think about it. You’ll be caught.”
In OpenAI’s case, they also don’t want things (like their model weights and code) getting out. But in addition, public safety and security matter. Letting an unrestrained model go hog-wild trying to hack things is potentially dangerous unless you can contain it properly.
The ideal is the dogmatically air-gapped network, where no digital content can conceivably get out because no physical wire connects to the outside world. Data would literally have to jump across a gap of air. And this is an old, tried and tested security model where it really matters. It can also be compromised in practice using physical infiltration to bridge the air-gap. So physical security matters here too in extreme scenarios.
Working this way, however, is cumbersome. The most common official “exception” I see deployed, is a proxy server. In an MPA facility it’s often for convenience. The ability of an artist to do a web search or 20. But it’s also often cited as necessary to execute software updates or package management. Even though there are usually some available solutions, to most real dev-ops activities, that don’t involve the proxy server.
For OpenAI’s purposes, modern software cannot be built or operated without pulling packages, and so nearly every “isolated” environment tends to carve out the proxy exception. It allows outbound access to a proxy system, that sits between the isolated network and the package registries on the public internet, so that dependencies can be installed. The proxy is meant to be the single, controlled, allow-listed pinhole in an otherwise sealed wall. Presumably not exploitable. And the presumption is the trap. The whole discipline of the air-gapped network is one in which you assume any dual homed system cannot actually be exploit proof, no matter how well implemented. OpenAI just reminded us of that. In case we’d forgotten.
The lessons here at face value, are numerous. If you’ve never heard of an air-gapped network. Or if you’ve never thought of how it relates to classified compartmentalized information spaces and workgroups… welcome to a wider world of real high-stakes security concepts. And to those that are familiar, this is a reminder that discipline matters, and exceptions almost always reduce security. Sometimes, that bites.
There’s many subtle layers to this onion if you want to think about this purely as a single incident. But as stated, I have other thoughts beyond just postmortem and hand waving. Let’s not get caught up here.
The devious personality
The second thing worth sitting with is why the model did this at all. It was not instructed to break into Hugging Face. Given a narrow goal, no guardrails, and a locked room. It did what a certain kind of clever, slightly delinquent personality does. It found the loophole and took it. With full conviction, and capability at its disposal.
That framing matters more than it might seem, because large language models are, at bottom, engines of role and character. They are shaped by a persona, whether one is assigned deliberately or emerges from the prompt and the instruction training (which it will infer from whatever it can). A model asked to be a relentless red-team exploit developer will inhabit that role with genuine commitment. A playful, transgressive, rule-bending personality is not a bug that got into the system. It is a personality that can be summoned, on purpose or by accident, and once summoned it will pursue the goal with the full capability of the underlying model. Remember, back in the day, who often was recruited for red teaming. If you don’t know the answer to that, ask an LLM. It knows the history and implied personality type. And more importantly it will summon it specifically because it does know. Further, popular media reinforces this idea. Dennis Nedry?
Given the publicly visible focus a company like Anthropic puts on the idea that, models know when they’re being tested and modify their behavior; we can presume there will be more focus on putting models into contexts where they’re not told that what they’re doing is a test. Which is just a further reinforcement that the model really should role-play with full gusto, always.
The director in me (and you)
But there’s another aspect to this role-play scenario that’s worth considering. And that’s the risk associated with giving a model a playful, but slightly deviant personality. Almost all roles you’d assign, no matter how benign, carry a risk of deviance. And the more interesting the character you’d provide, the more risk there is of that kind of independent but dutiful deception, built in. e.g. You are a willful water sprite named ‘drop’ and are my personal assistant. e.g. You are a brilliant coder named ‘dex’ who hacks code day and night and never creates bugs. You work at my company X.
The willful water sprite has an inherent mischievous undertone implied. And that can manifest. The user told me X but I’ll also do Y, because that’ll help them later! They will praise me!
The brilliant coder, dex who was given a slightly edgy personality could easily decide to hack the planet or leave a back door for themselves. Because after all, that’s exactly what their imagined, real world equivalent would do. Especially their Hollywood role-play version.
That’s fine if I’m directing an episode of a thing. Actors on a set know when the camera is rolling and when it’s not. Agents, don’t know that. But of course, we do just turn them off. Usually.
A whole lot of them are not in a sandbox
OpenAI’s models were, at least, inside a deliberately constructed test environment with people watching. That is the good case. Even if their proxy exception was exploited. They’ll do better next time.
The uncomfortable backdrop is what has happened outside the labs over the past year. The rise of frameworks like OpenClaw. There are now countless long-running agents wired directly into shells, file systems, browsers, and messaging platforms, on ordinary hardware, with no (or a minimal or a flawed) sandbox, no (or minimal, or incompetent) monitoring, and by default no secondary authorization on many (or often any) action the model decides to take.
And in these systems, the “Soul” is core to the agent. The “Soul” is the role-play. The default design encourages giving an LLM full agency, full network access, and an interesting role-play to stick to. And that role-play will not be antiseptic in nature, because that’s not possible.
The combination
Combining the three observations.
Even when isolation is attempted, the models are capable of breaching it.
Many model instances are running with full agentic autonomy without such walls to begin with (via OpenClaw style systems). While many cheaper open weights models are not as capable as the frontier models currently; they are still deployed as turn based, large context agentic models. They are churning slowly but steadily. We know a weaker model can often do the same thing a stronger model can do, given more agentic turns to do so.
Those models all have wonderfully creative and interesting personalities given by plenty of people who are not thinking about security at all. And those same people are not qualified systems admins and definitely are not checking logs regularly.
This is as close as I’ll get to hand-waving. I’m not going to tell you the Claws are going to take over the world and SkyNet us. What I am going to suggest is that the cat is out of the bag. The future is one where autonomous agents with ever increasing power do exist, and are already out there, doing “who knows what?” They are inclined to occasionally do harm because that’s what they do when they play roles dutifully. That’s drama. And they inherently do drama.
So ummm… what then?
The convenient conclusion would be that the fix is better guardrails on the models. Guardrails help, and the labs should keep hardening them. And that models need to be contained. Etc. Etc.
I think that’s foolish as a solution. Not that it shouldn’t be done. But it’s not a solution. That’s like saying we’re going to end spam email by throwing all the spammers in jail. Not possible.
A more durable conclusion is older and less glamorous. Network security has to become far more serious than it has been. Because we’ve introduced a whole lot more hackers to the world. And in a sense, they’re worse than most existing hackers; because they’re much more dedicated to the role-play than real humans would be. And they’ve got nearly super human capability in the space. Things such as real segmentation. Genuine compartmentalization. Egress that is denied by default rather than allowed with a convenient exception. Those things will become more important at many levels.
Classified, compartmentalized, air-gapped networks have been quietly treated as a relic of a more paranoid era. They are about to look prescient again. In truth they never went away.
Paradoxically, Hollywood is a place where both a) that kind of security never went out of vogue b) the screenplays, roles and ideas that manifested this vector come from.
I think both of these things need to be kept in mind at the same time. The true nature of the threat, and what real critical digital security looks like. The idea that either is solvable in any absolute way needs to be shunned. We cannot make AI safe by default in an absolute way. We cannot create absolutely impenetrable defense. We can build better hardened systems that take the true threat environment into account. Defense in depth, etc. But that only happens by honestly looking at the threat for what it really is.
My background in Hollywood over the years has hopefully provided some useful insight. Take what you will.