Sulat.com / Writing on AITuesday, November 17, 2026
Artificial Intelligence/17 November 2026

OpenAI signs Apollo, METR, Transluce and Goodfire to assess its models from inside

OpenAI says Apollo Research, METR, Transluce and Goodfire will get deep access to its internal models, training data and monitors. Assessors work on site, and OpenAI can redact confidential evidence from public reports.

Glossy 3D OpenAI logo in green glass and chrome beside a glass-walled room where four visitors with badges work at laptops.

At a glance

  • OpenAI says it has signed agreements with Apollo Research, METR, Transluce and Goodfire for the first phase of embedded assessments, giving outside experts deep access to internal models, training data and monitoring systems.
  • Apollo Research will put a team of six inside OpenAI for four months to assess four claims about whether models subvert their developers' controls, and plans two public reports, Apollo says.
  • OpenAI says assessors work on site on OpenAI devices, its board's Safety and Security Committee receives unredacted reports, and OpenAI can redact confidential information from public reports.
  • OpenAI says Apollo and Goodfire accept compensation that doesn't depend on results, while METR and Transluce accept none but may take API credits or compute.

OpenAI says it has signed agreements with four outside assessors, Apollo Research, METR, Transluce and Goodfire, for the first phase of its embedded assessments. Under the program, independent experts get deep access to OpenAI’s internal models and systems so they can assess alignment and test its safeguards.

OpenAI announced the program on X on 16 November, saying the assessments build on its alignment and safety research and bring independent scrutiny to how it identifies and manages risks.

What an embedded assessment is

OpenAI says assessors get some of the same internal resources its own researchers use: model checkpoints (saved snapshots of a model partway through training), source code, training and evaluation data, monitoring systems, technical documentation, and conversations with the researchers and engineers who did the work. The first assessments cover alignment and safeguards for models during training and internal use.

OpenAI says the program supports its commitments under the White House Accord on Super Intelligence, where it agreed to independent assessments of its safety controls.

Who is assessing what

Each team has a separate brief.

Four cards: Apollo hunts scheming, Goodfire debugs training, METR checks the largest run, Transluce tests alignment evals.
Each outside team has its own job, from hunting scheming to testing OpenAI’s alignment tests.
AssessorFocusScope
Apollo ResearchWhether models covertly work against their developers’ intentions, and whether OpenAI’s safeguards would detect itFour claims in two stages; a team of six for four months, per Apollo
GoodfireTraining run monitoring, debugging and alignment, using interpretability tools that analyse model internalsResearchers embedded at OpenAI; Goodfire will publish learnings as the work goes
METRMisaligned behaviour in OpenAI’s largest reinforcement learning run to date, including failures its tests and monitors may have missedA second, broader monitor assessment is preliminarily agreed, pending agreement on scope
TransluceThe quality and coverage of OpenAI’s post-training alignment evaluationsA follow-on of about eight weeks is planned for early 2027
The four pilot assessors, from OpenAI’s post and Apollo’s.

Reinforcement learning is the training method where a model is rewarded for outputs a grader likes. OpenAI says METR would set its own priorities within the agreed access and run its own searches, analyses and experiments. OpenAI also points to a three-week monitor red-teaming exercise it ran with METR, and to METR’s investigation of the Hugging Face incident.

What Apollo will assess

OpenAI lists four claims Apollo will assess in two stages: that training didn’t incentivise subversion, that strong evaluations and red teaming don’t elicit it, that models haven’t attempted it during internal use, and that the reasoning behind it would be detectable. For the first stage, OpenAI will provide selected public and unreleased models and checkpoints, relevant training data and evaluation results, training intervention records, monitoring systems and flagged behaviours, plus researchers who can give context.

Numbered list of four claims Apollo will assess, from training incentives to whether scheming reasoning would be visible.
Apollo’s first public report covers claim one; a second covers the other three.

Apollo’s own post says it will publish two public reports. The first covers whether training incentivises subversion, and the second covers evaluations, internal deployment and whether subversive reasoning would be detectable. All of Apollo’s work will happen on OpenAI hardware.

Apollo says OpenAI can withhold in-scope items for technical infeasibility, security risk or IP protection, but has agreed not to withhold anything simply because it’s unfavourable. OpenAI must tell Apollo that withheld items exist and give written reasons. Apollo can describe its access and publication terms publicly without OpenAI reviewing them first.

How the assessments run

Assessors who need sensitive confidential information work on site at OpenAI offices, on OpenAI devices, and follow at least the same security standards as OpenAI employees. Where direct access would expose model weights or architecture, OpenAI says it will consider alternatives such as pairing assessors with its researchers to run the analyses.

OpenAI’s Safety and Security Committee, an independent oversight committee of the OpenAI Foundation’s board, can see each assessment before, during and after it, and receives unredacted reports. Assessors and the committee can talk privately. Disagreements go first to OpenAI’s Safety Advisory Group and executives, who have a set time to resolve them, and then to the committee.

Flow: assessors work on site, write findings, then a public report OpenAI can redact and an unredacted copy for the board committee.
The public report can lose confidential evidence; the unredacted version goes to the board’s committee.

Conversations with employees are off the record, so they can inform an assessment but can’t be published or attributed. Assessors can ask OpenAI to attest to a statement they learned in conversation, and can publish what OpenAI attests to or say that it declined. Assessors keep independence in publishing, and OpenAI keeps the right to redact confidential information and to ask for responsible disclosure delays.

Apollo and Goodfire accept financial compensation that doesn’t depend on their results. METR and Transluce accept none but may take API credits or compute. OpenAI says it doesn’t believe compensation stops assessors being independent, and discloses it for that reason. It also asks staff at assessor organisations to observe a brief cool-off period before embedding with another AI developer.

What comes next

OpenAI expects the first phase to lead to longer-term embedded engagements that give continuous third-party safety signals on its most load-bearing safety claims and safety cases. It says it’s exploring future embedded assessments of its compliance with its safety frameworks and of its organisational culture and processes.

Apollo’s two public reports are the first outputs named in the posts, and OpenAI and Transluce share the intent to run their follow-on assessment in early 2027.

Sources

Disclosure: I’m an OpenAI, Firecrawl and Devin ambassador.


Discover more from Sulat AI & Tech News

Subscribe to get the latest posts sent to your email.