OpenAI releases CSAM detection classifier and adds ChatGPT warnings and temporary bans
OpenAI has released new tools to combat child sexual abuse material, including an image classifier that can detect CSAM images, including AI-generated ones, and new warnings and temporary account blocks in ChatGPT, TechCrunch reports. OpenAI developed the classifier with the National Center for Missing and Exploited Children and is giving select companies free access through its API, working with the Tech Coalition and ROOST to share it.
ChatGPT will now show an alert to users who try to access material that sexualises children, point them to specialised support and suspend the account for 24 hours as a warning; repeated offences could lead to permanent removal. OpenAI said it shaped the messages with organisations and researchers including Protect Children, the Lucy Faithfull Foundation and Joel Scanlan at the University of Tasmania.
The company is also publishing guidelines for developers using its tools to prevent, detect and report CSAM, and is giving law enforcement teams working on child safety AI-powered investigative tools, training and a field guide.
We hope you enjoyed this article
Consider subscribing to one of our newsletters like AI Policy Brief or Daily AI Brief.
Also, consider following us on social media:
More from AI Safety
Nov 16 OpenAI embeds six Apollo Research evaluators to test whether models try to subvert safety controls Nov 16 AI models abandon climate facts when users push back, UNESCO and UCL study finds Nov 16 OpenAI considered buying Anthropic-backed Goodfire for more than $5B, report says Nov 16 Anthropic says it has met its safety threshold for AI that speeds up AI research Nov 16 US and Russia set to oppose talks on binding autonomous weapons rules at UN meetingAI Policy Brief
Weekly report on AI regulations, safety standards, government policies, and compliance requirements worldwide.
Market report
2025 Generative AI in Professional Services Report
Thomson Reuters
This report by Thomson Reuters explores the integration and impact of generative AI technologies, such as ChatGPT and Microsoft Copilot, within the professional services sector. It highlights the growing adoption of GenAI tools across industries like legal, tax, accounting, and government, and discusses the challenges and opportunities these technologies present. The report also examines professionals' perceptions of GenAI and the need for strategic integration to maximize its value.
Read moreYou may also like
OpenAI Faces Proposed Class Action Over Children's ChatGPT Data
OpenAI safety reviewers faced months-long backlog
OpenAI Details Activation Classifiers for Model Safety
Texas Sues OpenAI Over ChatGPT Safety and Marketing
OpenAI Fixes ChatGPT Lockdown Mode Bypass
Daily AI Brief: the AI news that matters, in your inbox.