OpenAI Details Activation Classifiers for Model Safety

OpenAI detailed in a company blog post how it trains activation classifiers to detect unsafe or misaligned model activity. Starting with GPT-5.6, the company has run these classifiers on all assistant completions.

The lightweight models inspect internal activations and produce a score for each token as a response is generated. If a score crosses a threshold, OpenAI stops the response, sends the message to a reasoning model for review, and blocks tool calls until they are marked safe. The company says these classifiers are more efficient and more resistant to jailbreaks than larger classifiers that inspect text alone.

OpenAI typically trains the classifiers on up to 1 million training messages drawn from synthetic and de-identified conversations, with labels produced by reasoning models. The classifiers can detect only categories represented in labeled training data. They must also be retrained for each new pretrained base model, and their performance can decline as model weights diverge from the checkpoint used during training. The probes are one part of OpenAI's internal misalignment monitoring system, alongside model-based review, automated investigation and human response.