OpenAI says new safeguards cut reasoning-extraction attacks on GPT-6.2 Astra to 0.2%
OpenAI has set out how it fights adversarial distillation, in which an actor uses a frontier model's outputs without authorisation to train another model with similar capabilities but without the same safeguards, the company said on 20 November. It says this is a particular risk in dual-use areas such as cybersecurity and biology.
The company says it focuses on protecting its models' internal reasoning, or chain of thought, which can carry information the final answer leaves out and is especially valuable training data. In an internal adaptive red-teaming test, the average success rate of attacks that tried to recover reasoning from an earlier prompt fell from 54.0% against GPT-6 Astra with its original safeguards to 0.2% against GPT-6.2 Astra. With GPT-6.2 Astra and Sol, reasoning is now tied to the conversation that produced it and dropped if an earlier message is rewritten, and older model families can no longer read it.
In an internal study, a student model trained on exact reasoning traces from coding and defensive security work generally learned more cyber capability than one trained on the reasoning summaries users can see, which OpenAI says shows protecting traces can limit capability transfer. Its defences combine adversarial training, input and output classifiers, API controls and investigations of coordinated campaigns.
OpenAI says it has seen multiple unauthorised distillation campaigns this year, with weekly enforcement actions ranging from thousands to tens of thousands of accounts and rising sharply over the summer. It calls adversarial distillation a national security risk and notes it briefed Congress in February on activity indicating that DeepSeek was attempting to distill US frontier models.
We hope you enjoyed this article
Consider subscribing to one of our newsletters like Cybersecurity AI Weekly, AI Policy Brief or Daily AI Brief.
Also, consider following us on social media:
More from Cybersecurity
Nov 20 Army taps OpenAI first to bring AI into Project Gatekeeper software vetting Nov 20 Taylor & Francis retracts paper that hid a prompt telling AI reviewers to praise it Nov 20 GAO says AI magnifies cyberattacks and sets out 16 policy options for AI in cyber defence Nov 20 Dragos seeded two million AI prompts with secrets and found breaches at three providers Nov 19 Meta's Muse AI agent mixed up conversations and exposed private data, employees sayMore from AI Safety
Nov 20 Australia and China agree to hold talks on AI after Albanese meets Li Qiang Nov 20 METR says evidence cannot rule out drastic AI R&D acceleration within a couple of years Nov 20 Trump says he and Xi agree AI needs safety rules Nov 20 Dragos seeded two million AI prompts with secrets and found breaches at three providers Nov 20 Protesters rally against Anthropic in New York and San Francisco on its stock market debutCybersecurity AI Weekly
Weekly newsletter about AI in Cybersecurity.
Whitepaper
Tensordyne Napier: What If One Rack Could Do the Work of Nine?
Tensordyne
This Tensordyne whitepaper presents Napier, an inference-focused AI processor and rack-scale system based on the company’s TDN Math logarithmic number system. It examines infrastructure requirements for large mixture-of-experts and agentic models, compares major inference architecture approaches, and details the TDN AIP processor, TDN72 pod, TDN Link fabric, and Napier Ultra configuration. The paper reports simulation-based performance, cost, and accuracy-validation results, including Tensordyne’s projected comparison of one Napier rack with a nine-rack Nvidia Rubin plus Groq deployment; the chip is reported as taped out and in fabrication.
Read moreYou may also like
OpenAI says automated red teaming made GPT-6 Astra harder to jailbreak
OpenAI Expands Monitoring for Frontier Model Misalignment
OpenAI Finds Model Training Increases Chain of Thought Controllability
OpenAI Finds GPT-6 Astra Underperforms on Safety Research
DeepSeek and Moonshot reportedly used GPT-6 Astra outputs for training
Daily AI Brief: the AI news that matters, in your inbox.