Meta Commercial Terms let it crawl business websites for AI from October 30

The new Meta Commercial Terms, which take effect on October 30, 2026, include a crawling clause. It lets Meta, or anyone acting for it, use scrapers, bots and other automated tools on websites linked to a business’s products or services. Nieman Lab first reported the change on October 15. Search Engine Journal compared the old and new texts a day later.

Meta’s terms cover every business Page, not just advertisers

The terms apply to any business use of Meta’s products. That means running ads, but also managing a Facebook Page, an Instagram account, Messenger or a WhatsApp channel. A company that never buys an ad is still covered.

The current Commercial Terms, dated March 6, 2024, have no crawling clause. The Self-Serve Ad Terms from August 22, 2025 only let Meta crawl advertisers’ domains and URLs. The new version of the ad terms gets the same wider permission, also from October 30. There is no opt-out. A business that keeps using Meta’s products for business accepts the new text.

Meta may create test credentials to reach pages

The clause lists what Meta may do with the content: develop, improve, research and train its products. It also lets Meta “create test credentials” to access pages. Neither text mentions robots.txt, the file sites use to tell bots what they may read.

James Grimmelmann, a law professor at Cornell, told Nieman Lab the wording could let Meta train AI on news and pose as a user to pass registration walls. Matt Topic, a media lawyer at Loevy & Loevy, said it is unclear whether a court would enforce the clause, so publishers should assume it would. Benjamin Sobel of the University of Wisconsin compared the link between Page access and scraping rights to “tying” in antitrust law.

Andy Stone says the update is not about news sites

Meta spokesperson Andy Stone called Nieman Lab’s summary false and said the update is “not about scraping news sites.” Meta told Nieman Lab the terms only gather crawling language that was spread across several documents. It said they will not change how it crawls publishers, and that robots.txt still works. Meta declined to say whether test credentials could get past paywalls.

Google has taken a different route with news sites. Its AI contribution pilot pays publishers for content used in its AI features.

Meta-ExternalAgent can be blocked on its own

Meta’s crawler documentation lists separate bots. Meta-ExternalAgent collects content for AI training. Meta-WebIndexer feeds Meta AI search. A site can block the first and still allow the second, much like blocking a single Google crawler. Two other bots may skip robots.txt: Meta-ExternalFetcher when a user asks for a page, and FacebookExternalHit for security checks.

Before October 30, open your site’s robots.txt and check whether it names Meta-ExternalAgent. A site with no rule for that bot already lets it in. The legal experts quoted by Search Engine Journal say the new terms may give Meta grounds to argue that a block does not apply to its business users.