Sulat.com / Writing on AISaturday, November 21, 2026
Artificial Intelligence/21 November 2026

OpenAI says new safeguards cut reasoning-extraction attacks on GPT-6.2 Astra from 54% to 0.2%

OpenAI says its new safeguards cut the success rate of attacks that extract protected reasoning from GPT-6.2 Astra from 54.0% to 0.2%. Here is how adversarial distillation works, in plain words.

Glossy green 3D OpenAI logo beside a glass distillation flask dripping through a padlock into a small flask

At a glance

  • OpenAI defines adversarial distillation as using a frontier model's outputs without authorisation to train another model that can do similar things without the same safeguards.
  • In an internal study, a student trained on a teacher's exact reasoning traces generally beat one trained on reasoning summaries on cyber benchmarks, so OpenAI guards those traces.
  • OpenAI says its adaptive black-box test saw extraction attack success fall from 54.0% to 0.2% (earlier request) and 22.9% to 0.9% (current request) on GPT-6.2 Astra.
  • Defences work at two levels: request-level training, classifiers and API controls, plus campaign-level investigations that ban whole account networks.
  • OpenAI says its weekly enforcement against distillation campaigns has covered thousands to tens of thousands of accounts, and it plans to share more soon.

OpenAI has published a research post on adversarial distillation, which it defines as using a frontier model’s outputs without authorisation to train another model that can do similar things without the same safeguards. The post, dated 20 November, explains how OpenAI detects and disrupts it, describes an internal study on how cyber skills transfer between models, and reports that extraction attacks against GPT-6.2 Astra’s protected reasoning now succeed 0.2% or 0.9% of the time, depending on the attack, down from 54.0% and 22.9% against GPT-6 Astra with its original safeguards. This piece walks through the post in plain words, with each technical term explained where it first appears.

The short version

  • The problem: a second model can learn advanced skills from a frontier model’s outputs without inheriting the rules that limit how the first one can be used.
  • The focus: OpenAI is protecting its models’ exact reasoning traces, the step-by-step working a model does before it answers, because those traces carry more than the final answer does.
  • The defences: request-level safeguards (training, classifiers and API controls) plus campaign-level investigations that ban whole networks of accounts.
  • The result: in OpenAI’s internal test, the average success rate of extraction attacks fell from 54.0% to 0.2% for reasoning from an earlier request, and from 22.9% to 0.9% for reasoning from the current one.

What distillation is, and why OpenAI calls it a safety issue

Distillation is a common machine-learning technique in which one model learns from data another model produced. The teacher is the model that produces the data and the student is the model that learns from it. A student can learn from the teacher’s answers, from its reasoning, or from its judgements about which answers are better and why.

The trouble starts when that happens without permission. Setting intellectual property aside, a student trained this way can pick up frontier skills without inheriting the safeguards that govern how the original model can be used, and OpenAI treats that as a safety problem. The worry is sharpest in dual-use areas such as cybersecurity and biology, where the same skill that helps a defender can help an attacker. OpenAI’s example: an adversary distils a model’s cyber skills and uses the resulting model to help carry out cyberattacks that the original model would have refused to assist with.

Teacher model copies to student model; exact reasoning traces are protected, so the student gains skill without safeguards
A student model can copy skill from a teacher without copying its safeguards.

Why exact reasoning traces matter

OpenAI puts particular weight on protecting what it calls protected reasoning: the intermediate steps a model generates while it works toward an answer, also known as its chain-of-thought. These traces can hold information the final answer leaves out, which makes them especially useful training data for another model.

OpenAI ran an internal study to see how much that matters for cyber skills. The baseline was what users normally see: final responses, reasoning summaries and tool interactions. Every training run started from the same pretrained student model. OpenAI built a supervised fine-tuning dataset, which means training on worked examples, made mostly of trajectories from Astra-level teachers on coding and defensive cybersecurity tasks. Each trajectory holds every step the teacher took, including tool use and the final response. It then tested each student after that training and again after a light round of reinforcement learning in cybersecurity and coding environments.

After supervised fine-tuning, the student trained on exact reasoning traces generally beat the student trained on reasoning summaries. After reinforcement learning, the exact-trace student still scored higher at the largest inference budget tested on both benchmarks, ExploitBench and SEC-bench Pro, while the summary-trained student scored higher at lower budgets. OpenAI notes that Anthropic has reported a similar advantage for full reasoning transcripts over summaries on SWE-bench Verified, and says its own result extends that advantage to frontier cybersecurity.

The takeaway is blunt. Guarding reasoning traces can limit how much cyber skill moves to a student, even when every task in the data is one OpenAI’s usage policies allow. Spread across many harmless-looking interactions, exact traces teach a student skills that reach beyond the tasks the teacher actually did.

Two layers of defence

OpenAI describes its defences as two layers that reinforce each other. The request layer limits what any single conversation can give away. The campaign layer looks for coordinated efforts across many accounts. Attacks found in investigations then become training and test examples for the request layer.

Campaign-level detection above request-level safeguards: refusals, classifiers and API protections for reasoning
OpenAI guards each request and hunts coordinated campaigns, and each feeds the other.

Defensive training. Adversarial training teaches models to recognise attempts to extract reasoning and to refuse to disclose protected reasoning, while still answering legitimate requests for explanations and help. The training examples come from successful attacks found through OpenAI’s automated red-teaming, its investigations and responsible disclosures from outside researchers and industry partners.

Classifiers. A classifier is a model that checks text and flags it. OpenAI uses input classifiers to spot attempts to extract protected reasoning and output classifiers to spot leaks of it in assistant messages and in tool-call arguments, for both streaming and non-streaming responses. Tool calls need their own coverage because a model can expose protected reasoning inside a tool’s arguments without putting it in the final answer, and output classifiers can block a suspected leak before it reaches the caller.

API protections. These limit how protected reasoning can be reused across conversations and models. OpenAI’s September disclosure described attacks that moved encrypted reasoning from one conversation into another and asked the model to show it in readable form. OpenAI says it strengthened protections across user, workspace, organisation and model-family boundaries, including closing a pathway that could expose another user’s encrypted reasoning.

With GPT-6.2 Astra and GPT-6.2 Sol, OpenAI also ties a model’s protected reasoning to the conversation history that produced it. As new messages are added, the model can build on its earlier reasoning. If an earlier message is rewritten, the API drops the reasoning that followed it. Older model families with weaker safeguards are blocked from reading Astra and Sol’s protected reasoning. For the wider launch, see our earlier piece on GPT-6.2 Astra and Sol.

Campaign detection. When its systems detect an unauthorised distillation campaign, OpenAI investigates the accounts responsible and enforces against the whole network, consistent with its policies and terms of service. Each investigation yields insights that sharpen detection, and OpenAI shares indicators of compromise, the digital fingerprints of an attack, with other model providers so they can look for related abuse on their own services. Its consistent-standards policy sets shared safeguard and enforcement standards for providers that offer its models.

The numbers

To measure how well the request-level safeguards work together, OpenAI tests whether attackers can still extract protected reasoning in a setup that closely mirrors production. Static evaluations replay a fixed set of attacks that worked before. Adaptive evaluations use automated AI red-teamers that change their approach based on how the system responds, which can find weaknesses static tests miss. In black-box testing the attackers get the same view an outside user would have, and in white-box testing they get extra information about the system.

The figures below come from adaptive black-box testing, averaged across six attacker models. OpenAI tested whether attackers could recover reasoning the model produced for an earlier request, or for the current one.

Bars: attack success 54.0% to 0.2% for earlier requests and 22.9% to 0.9% for current requests on GPT-6.2 Astra
In OpenAI’s test, extraction attacks fell from 54.0% to 0.2% and from 22.9% to 0.9%.
Attack typeGPT-6 Astra, original safeguardsGPT-6.2 Astra, updated safeguards
Extract protected reasoning from an earlier request54.0%0.2%
Extract protected reasoning from the current request22.9%0.9%
Adaptive extraction attack success rate, black-box, averaged across six attacker models. Lower is better. Source: OpenAI.

OpenAI says it’s now applying selected safeguard improvements to GPT-6 Astra as well.

What the campaigns look like

Campaigns aren’t rare. OpenAI has seen several unauthorised ones on its platform this year, and its weekly enforcement against them has covered anything from thousands to tens of thousands of accounts, with the rate climbing sharply over the summer. To track progress it measures how common distillation abuse is and how long it takes between detecting an attack and enforcing against it. The goals are to limit the volume and value of the data distillers can collect, raise the cost of running a campaign and cut the time it takes to respond to new attacks.

The September report has the specifics. It put the start of one campaign on 1 July, with spikes on 24 and 25 July of 16,000 requests from more than 4,000 users, and a wider cluster of more than 15,000 users that OpenAI says it fully disrupted by 28 July. OpenAI attributed a core cluster of that activity to individuals associated with Moonshot AI, the developer of Kimi. In the new post it adds that it plans to share more soon about recent campaigns it has disrupted.

The national security argument

OpenAI calls adversarial distillation a national security risk as well as a safety one. Its reasoning: a foreign actor that reproduces advanced capabilities from a US model can deploy them without the protections the developer built in. The post says OpenAI briefed Congress in February on activity indicating that DeepSeek was attempting to distil US frontier models.

OpenAI says it supports legislation that would give the US government more tools to identify and sanction foreign actors that illicitly extract capabilities from American AI models. It has also told Congress it supports the Chip Security Act, the AI OVERWATCH Act and the MATCH Act. Its argument is that export controls and distillation defences work together: controls limit adversaries’ access to training hardware, and distillation defences make it harder to acquire some of the same capabilities from models that already exist. Both, it says, help maintain a US lead in frontier AI and give developers more room to slow a release when they need time to test safeguards.

What OpenAI tells other developers to do

Supporting the safe spread of frontier capabilities is one goal of this work, so OpenAI lists four steps other developers and platforms can take:

  • Protect private reasoning across the full interaction. Check assistant messages and tool-call arguments for leaks, and restrict how encrypted reasoning can be reused across conversations and models.
  • Keep successful extraction attack logs from red-teaming, production monitoring or responsible disclosure as regression tests. Where privacy policies allow, use them to improve training and classifiers, and replay the original attacks against each updated defence.
  • Combine request-level mitigations with campaign-level investigation and enforcement.
  • Coordinate safeguard requirements and enforcement with deployment partners, and share expertise with other developers, to keep a high floor of distillation safeguards across the industry.

OpenAI says it has also shared findings through the Frontier Model Forum and government information-sharing channels, keeping back the details attackers could exploit.

Who this affects

For developers who build on OpenAI’s API, the visible changes are in how reasoning behaves. Protected reasoning is tied to the conversation that produced it, so rewriting an earlier message drops the reasoning that followed. Anyone who edits conversation history mid-session should read the keep-prefix-unchanged guide OpenAI links for that behaviour. For other model providers, the post is a checklist of what OpenAI found worked. For everyone else, it shows why a lab treats copying a model’s outputs as a security problem and what OpenAI is doing about it.

The terms in one place

TermIn plain words
DistillationTraining one model on another model’s outputs.
Adversarial distillationDoing that without authorisation, so the student skips the teacher’s safeguards.
Protected reasoningThe step-by-step working a model does before it answers, also called chain-of-thought.
ClassifierA model that checks text and flags it, here to catch extraction attempts and leaks.
Red-teamingAttacking your own system on purpose to find weaknesses.
Attack success rateThe share of attacks in a test that worked. Lower is better for the defender.
Indicator of compromiseA fingerprint of an attack that other providers can search for.
Definitions are mine, written to match how OpenAI uses each term in its post.

Disclosure: I’m an OpenAI, Firecrawl and Devin ambassador.


Discover more from Sulat AI & Tech News

Subscribe to get the latest posts sent to your email.