Travis Lelle PRO

@Revertron I've wondered the same thing, and I tend to believe it's the latter. Setting up these runs for research is costly and time consuming. I'm sure they always expect some unpredictable actions, but I don't believe they follow live telemetry of what the agent does in real time for any substantial period of time when running their tests, and running agents in parallel to make the most of research runs only contributes to burying one agents actions amongst many others in real time. They've also been running GPT-Red to adversarially train their public models, rewarding defenders when they complete tasks and ignore attacks, so it's unclear whether this incident is part of that larger research or not.

HuggingFace included what they're hardening in their environment to mitigate future attacks, and OpenAI wrote a press release outlining what they're doing to prevent this from happening again. That sounds like they're doing their job to me.

A good gateway setup can capture most of the telemetry needed to trace an agent's actual execution and calls (and not so much the reasoning piece, which doesn't always directly correlate to the action taken): gateway the model calls, gateway the tool calls, propagate one ID through both, hash the context spans, then add kernel-level exec capture. The reasoning layer comes last and buys the least.