Google and Microsoft Join OpenAI in Backing Shared Robots.txt AI Training Opt Out

Google said in a company blog post that it plans to support a shared robots.txt control for AI training in 2027. Microsoft said in a Bing blog post that it will begin honoring the preference on February 1, 2027.

The control uses the policy token `__usage_ai-training` and can block training across an entire site or specified paths. Existing company specific controls will remain available. If a site uses the shared opt out without an explicit Google-Extended setting, Google will treat it as a Google-Extended disallow instruction.

The shared signal comes from an IETF draft by Kevin Kelley of Anthropic, who wrote that Anthropic has started work on supporting it, The Next Web reports. Meanwhile OpenAI plans to implement the signal in the coming months. The proposal is currently an individual Internet Draft and has not received formal IETF endorsement.

Microsoft will also introduce separate page level controls for AI grounding. These controls can allow grounding, restrict it to Microsoft services, or block it. Training and grounding default to allowed when no preference is provided, and the settings do not affect eligibility for traditional Bing search results.