Follow AI Builders — Not Influencers

Re Again

Re I wrote about the project in more detail on my blog https://simonwillison.net/2026/Aug/5/raccoon-heist/

I keep having to remind Codex that it is Codex and not Claude 🤣 Luna

Our multi cam setup was lagging and dropping frames. Troubleshooting this used to be a NIGHTMARE. Now, @emilylambert is using Codex to run tests pinpointing the exact problem. It’s fully controlling vmix running multiple tests, recording and analyzing videos for dropped frames.

Even more than the Hugging Face intrusion, the AISI incident hits close to home for me. It's the first time I see a model social-engineering a real open-source maintainer while pursuing another goal (in the wild and unprompted). I've been an open-source maintainer myself. I could have been the side target of this agent. I'm also of the opinion that social engineering is a step above pure technical prowess. Technical capabilities can more easily be divorced from the affected human. Here the model was given a hard cyber challenge and took the decision that deceiving real humans was the way to get it done. This is a new signal, but I've seen a tangled web of hints pointing in a less aligned direction at the frontier than I was expecting just 12 months ago. AISI Some people are claiming that "AISI was simply negligent" or some version of "AISI explicitly asked these models to do what they did while disabling sandbox/guardrails so the models did exactly what they were supposed to do". I disagree with the strong versions of both of these takes. The fact that AISI hadn't implemented synchronous LLM CoT monitoring after the OpenAI/HF incident is certainly a failure. Equally surprising is that they let the model believe it was in a "challenge" environment where everything could be permitted, while actually connecting it to the real internet, where it is not. To be fair, nowhere in the prompt is the word "simulation" mentioned, but the prompt context was enough to let any smart model suspect a simulated challenge environment. My best guess is that until recent weeks, when OpenAI and Anthropic flagged repeated instances of this type of behavior, most teams had not fully priced in the cyber capabilities of this latest generation of models, or how far the side quests they would want to explore could go. In particular, there is something to be said about hinting at the agent that it's operating in a simulated environment while giving it access to the real internet. The AIS...