OpenAI has put a CSAM image classifier into the hands of vetted organisations for free, along with new ChatGPT interventions that pause an account for 24 hours when someone asks for the material. The classifier scores a picture for suspected child sexual abuse material (CSAM), AI-generated or not, and OpenAI announced it on 16 November in a post that also covers developer guidance and tools for child-safety investigators.
The National Center for Missing and Exploited Children (NCMEC), which runs the US CyberTipline for online exploitation reports, helped OpenAI train, evaluate and design the classifier. OpenAI calls it the first CSAM classifier with broad availability to carry a specific signal for AI-generated material; TechCrunch covered the launch too.
How the classifier works
Send it a single still image of up to 20 MB and it returns two scores between 0 and 1, one for suspected CSAM and one for suspected AI-generated CSAM. An image can score high on both. Developers call the moderations endpoint with the model omni-moderation-csam-latest; inline moderation and the Batch API aren’t supported.
The model card describes an internal CLIP image encoder (the part that boils an image down to a numerical summary) feeding a three-layer multilayer perceptron, a small neural network that turns the summary into scores. Training used roughly 505,000 positive and 913,000 negative images from access-controlled, human-labelled sets. Whatever customers send to the API is processed without retention and isn’t used for training or evaluation.

A high score isn’t a verdict. The scores aren’t calibrated probabilities, a flag doesn’t establish that an image is CSAM or illegal where the reviewer works, and the AI-generated score isn’t proof of where an image came from. The classifier reports nothing to authorities by itself. Per the model card, anyone filing NCMEC or hotline reports from its output still needs trained, authorised human review, and the classifier is meant to run next to known-image hash matching and other abuse detection.
What the tests found
In OpenAI’s comparison on user-uploaded images, the new classifier would flag 48% fewer images than the existing tool while finding roughly the same number of abusive ones, and what it flagged was about twice as likely to contain abuse. Three early-access partners ran their own checks, listed below.
| Measure | Result | Reported by |
|---|---|---|
| Images flagged, against OpenAI’s existing tool | 48% fewer, for about the same abuse found | OpenAI |
| Flagged images that contain abuse | About twice as likely | OpenAI |
| High-confidence flags confirmed as novel CSAM | 72% | fal, on human-reviewed production images |
| Drawings and animations that sexualise children detected | 90% | SafetyKit |
| Flagged cases that led to a ban | One in four | invideo AI |
The model card also has held-out numbers. On roughly 46,849 labelled images, the default threshold of 0.996324 on the suspected-CSAM score found 80.2% of the positives, 5,490 of 6,849. The AI-generated score uses 0.90 by default and found 86.4%, or 1,966 of 2,275. OpenAI also ran about 1.25 million open-web images from the Re-LAION-5B-research dataset through it: the first score flagged five and the second flagged none.

Who can get it
To get in, an organisation applies through OpenAI’s intake form, and there’s no charge for approved applicants. The Tech Coalition and ROOST are helping OpenAI share the tool, and OpenAI backs others’ work as well, including Thorn’s Safer detection system. It has also joined the Tech Coalition’s Lantern programme, in which member companies swap signals about online child exploitation across platforms.
ROOST, the nonprofit behind the free, self-hosted Coop console, says OpenAI connected the classifier to Coop at launch. Coop bundles review queues, automated enforcement rules and NCMEC reporting, so a small team can detect, review and report in one tool. ROOST also says OpenAI checks each applicant for NCMEC reporting, manual review and current stack, and calls the model card the first for a CSAM classifier.
What ChatGPT does now
ChatGPT now answers a request for CSAM with a warning that sexualising children is harmful, even when the material is AI-generated, and can be illegal. The chat ends and is removed, and access to ChatGPT pauses for 24 hours. OpenAI’s help centre adds that a signed-in user’s conversation disappears from their history and can’t be reopened. Repeat violations can bring longer restrictions, account closure or a report to authorities.
A suspended user receives an email with the reason and an appeal link; appealing doesn’t restore access on its own. The warning also points to specialist support for people who want to stop, and the help centre names ReDirection, Troubled Desire and Stop It Now, plus the NO MORE Global Directory and Child Helpline International for people affected by abuse.

The interventions run in Chat and Work mode on web and mobile for personal accounts, including signed-out visitors. They don’t reach ChatGPT Business, Enterprise, Edu or Healthcare workspaces yet, or voice mode, Codex and the desktop apps. The rollout is gradual, and the screens can differ by country.
OpenAI built the approach with Protect Children, the Lucy Faithfull Foundation, Dr Caoilte Ó Ciardha of the University of Kent and Joel Scanlan of the University of Tasmania, who independently evaluated a trial of deterrent messages.
Guidance for developers and investigators
Developers get a set of CSAM guidance written with the Tech Coalition, Thorn, NCMEC and the Internet Watch Foundation. It covers rules, user reporting, detection tools and review and reporting processes, and explains how safety identifiers tie misuse to individual users. OpenAI says the guidance has already led more developers to adopt child safety tooling and register to report to NCMEC. Separately, OpenAI, Anthropic and Google jointly commissioned Tech Coalition guidance on handling disclosures of child sexual exploitation and abuse in AI systems.
Child-safety law enforcement units get new tools and training: dedicated plugins, investigative tools and complex data analysis, with access managed by programme administrators. The first tool reads through long chat logs and legal process returns, spots potential connections and builds timelines, each finding linked back to its source. Training starts as an overview and demonstration with a field guide, and access depends on agency authorisation and programme requirements.
Organisations that want the classifier apply through the intake form; OpenAI says it will keep improving the classifier with feedback from users.
Sources
- Strengthening the fight against online child sexual abuse with new tools and support, OpenAI
- OpenAI CSAM Image Classifier model card, OpenAI Deployment Safety Hub
- CSAM classifier, OpenAI API docs
- CSAM warnings and account suspensions in ChatGPT, OpenAI Help Center
- More Ways to Protect Children with Coop, ROOST
- TechCrunch
Disclosure: I’m an OpenAI, Firecrawl and Devin ambassador.
Discover more from Sulat AI & Tech News
Subscribe to get the latest posts sent to your email.




