OpenAI Will Honor Shared robots.txt Signal for AI Training Preferences

Oct 30, 2026
OpenAI will support a shared robots.txt signal that lets website operators communicate AI training preferences to multiple providers. The company plans to implement the signal in the coming months.

OpenAI will honor a shared robots.txt signal for foundational generative AI training preferences, the company said in a website post. The proposal uses the policy token __usage_ai-training and was authored by Kevin Kelley of Anthropic, according to Unite.AI.

Website operators can use standard Allow and Disallow rules to express preferences for specific paths. The signal adds no new directives or grammar, and existing parsers can ignore policy groups they do not recognize.

OpenAI plans to implement the signal in the coming months and encouraged other AI providers to honor it. Its current controls for individual crawlers will remain available and continue to apply. The proposal is an individual Internet Draft and does not have formal endorsement from the Internet Engineering Task Force.

We hope you enjoyed this article

Consider subscribing to one of our newsletters like AI Policy Brief, Sales & Marketing AI Weekly or Daily AI Brief.

Also, consider following us on social media:

AI Safety & Regulation AI Brief AI Brief (X)
Free newsletter

AI Policy Brief

Weekly report on AI regulations, safety standards, government policies, and compliance requirements worldwide.

Guide
Hosted Speech Recognition APIs

Hosted Speech Recognition APIs

Market report

2025 Generative AI in Professional Services Report

Thomson Reuters

This report by Thomson Reuters explores the integration and impact of generative AI technologies, such as ChatGPT and Microsoft Copilot, within the professional services sector. It highlights the growing adoption of GenAI tools across industries like legal, tax, accounting, and government, and discusses the challenges and opportunities these technologies present. The report also examines professionals' perceptions of GenAI and the need for strategic integration to maximize its value.

Read more

You may also like

Free, six days a week

Daily AI Brief: the AI news that matters, in your inbox.