Home
Friday, 18 September 2026 WARSAW 10:15 NEW YORK 04:15
AI · Search · Performance · E-commerce
Just in

Pew: people in 34 of 37 countries expect AI job losses. Young adults worry most

SEO & AI search

Google says blocking Mediapartners-Google now touches Ad Manager, not just AdSense

Google widened the description of its Mediapartners-Google crawler on 17 September 2026. Rules aimed at it now affect Ad Manager, not only AdSense.

Grzegorz Kubicki 18 Sep 2026, 09:10 reported from 1 sourcesGoogle crawler documentation
Code editor on a monitor with a file tree open on the left

Photo: Daniil Komov / Pexels (Pexels License)

Google rewrote one line about the Mediapartners-Google crawler on 17 September 2026. The line used to name AdSense alone. It now says crawling rules for that crawler “affect various ad-related Google products”, and gives AdSense and Google Ad Manager as examples. The page carries no other announcement.

The old sentence named AdSense, the new one names Ad Manager too

The earlier text told publishers that rules addressed to Mediapartners-Google affect Google AdSense. That was easy to act on. A site that had left AdSense years ago could block the crawler and expect nothing else to break. The new wording removes that certainty. Ad Manager is now on the list, and the list is open.

A star in robots.txt does not reach this crawler

Mediapartners-Google ignores the global user agent. A blanket rule written for every bot does not apply to it. The crawler has to be named on its own line. That detail matters more now, because the same line reaches further than it did. Publishers who wrote their file once and never opened it again may be blocking more than they think.

AdSense in-article 336x280

AdMob and Local Services Ads are still not named

Google gave two examples and stopped there. The documentation does not say whether Google Marketing Platform, Local Services Ads or AdMob sit behind the same crawler. Roger Montti spotted the change and wrote it up in Search Engine Journal on 17 September 2026. Google has not explained the gap.

Publishers already have enough moving parts in ad measurement

The change lands on publishers who are already rechecking their setup. Google recently said AdSense will stop counting ads that never render. Chrome also added four experimental ad metrics to its public dataset. Each of these touches the same revenue line from a different side.

From 17 September 2026 one line in robots.txt carries more weight than it did the day before. A rule written to switch off AdSense now reaches Ad Manager as well. Google published the date of the documentation edit, not a date when the behaviour changed, so the safe reading is that it already applies.

Sources

  1. Google crawler documentation — Source document: Google's own page on special-case crawlers, updated 17 September 2026\nSearch Engine Journal

Published by IMARKETING Sp. z o.o., Al. Jana Pawla II 150/127, 31-864 Krakow, Poland. Company register (KRS) no. 0001203282, VAT ID PL6751820741.  ·  Editorial office: Al. Jana Pawla II 150/127, 31-864 Krakow, phone +48 516 110 813, editor@marketingnewsroom.com.  ·  Editor-in-chief: Grzegorz Kubicki.  ·  Corrections: editor@marketingnewsroom.com.