Google and Microsoft back one robots.txt rule to block AI training

Google and Microsoft said on 30 October 2026 that they will honour a shared AI training opt-out in robots.txt. One rule will let a website tell several AI firms at once not to train models on its pages. The format comes from the AIPREF working group at the Internet Engineering Task Force (IETF), the body that writes internet standards.

Google maps the shared rule onto Google-Extended

Google posted the plan on its Search Central blog. If a site uses the shared rule to opt out and does not mention Google-Extended, Google will treat it as a block on Google-Extended. That is the training control Google has offered since 2023. Company rules keep working, so a site can block all AI trainers and still allow Google. Google said sites that allow Google-Extended can be cited in Gemini app answers and get clicks from them.

Google plans to support the rule in 2027, together with other AI providers. It tied the move to the EU code of practice for general-purpose AI models, which names robots.txt as a main way for publishers to state their choices.

Microsoft splits training from AI answers in Bing

Microsoft went further on the Bing blog. The shared rule uses the user agent __usage_ai-training in robots.txt. Bing adds a page-level training tag and a separate grounding tag. Grounding means using a page to build an AI answer in Copilot, Bing or Microsoft 365. A third option, DisallowExtended, keeps a page in Microsoft’s own AI products but out of the grounding services it sells to other firms.

Microsoft warned that blocking grounding will cut a brand’s visibility in AI answers. Normal Bing rankings do not change. Search Engine Roundtable reported that OpenAI and Anthropic also back the shared training rule. The news follows weeks of fights over crawlers: Meta put its new commercial terms on hold after publishers objected, and US publishers have asked Congress to unmask hidden AI bots.

Bing sets 1 March 2027 as the deadline

Bing will start reading the new tags on 1 February 2027. They replace NoArchive and NoCache, the old tags Bing treated as AI limits. Sites that use those tags today have until 1 March 2027 to publish the new ones if they want to keep their limits. A temporary grounding setting opens in Bing Webmaster Tools on 15 December 2026 and expires on 30 September 2027.

Open your robots.txt file and your page templates today and search for NoArchive and NoCache. If you find them, they stop working as AI limits on 1 March 2027, and the new training and grounding tags need to be in place before then.