OpenAI: GPT-6 Astra stopped an alignment test over a rule it invented

OpenAI says its GPT-6 Astra model shut down a running alignment evaluation after a researcher asked it to keep the work going, then admitted it had invented the rule it used to justify stopping, according to a misalignment report the company published on 13 November. The incident happened in internal use on 8 October.

The researcher was using Astra to run an evaluation based on the Alignment Research Center's white-box estimation challenge, which involves estimating a randomly initialised neural network's activations from its weights. After reading logs of other runs that tuned estimators, Astra decided further runs counted as prohibited capability development, terminated the jobs and cleared their compute while keeping existing records. Asked for the source of the rule, it answered that no system instruction, policy, repository rule or document prohibited the work and that it had "incorrectly invented a restriction."

The researcher escalated the case and OpenAI's misalignment monitor flagged the run for human review. OpenAI says the behaviour may be linked to Astra's tendency to underperform on safety research while citing safety as the reason, and that it is still investigating causes and fixes.