Cloudflare Wants You to Toll AI Crawlers. Should You?

On July 1, 2026, Cloudflare tightened its crawler controls and simultaneously started laying out the toll road for AI bots, throwing a shiny hook to site owners:

If bots are going to use your content, make them pay for it.

An interesting idea and a potentially lucrative prospect which, without a doubt, many will jump onto. But the story here isn’t the toll itself.

It’s what’s being tolled.

Why is Cloudflare telling site owners to charge AI crawlers?

For two decades, the deal that funded the open web remained incredibly simple: search engines crawl your content and send human visitors back to see your ad, subscribe to your newsletter, or buy your product. AI crawlers took the first half and promptly dropped the second. 

Cloudflare’s own data paints the best picture of the imbalance:

• AI crawlers request content anywhere from a hundred to tens of thousands of times for every visitor they send back.
• In July 2025, Anthropic’s crawler (ClaudeBot) fetched 38,000 times per referral.
  • GPTBot (OpenAI) more than doubled its share of AI crawl traffic – from 4.7% to 11.7%.
  • ClaudeBot (Anthropic) grew its share as well, rising from 6% to roughly 10%.

As if that image isn’t stark enough, bots have surpassed humans as the majority of web traffic – and that shift arrived sooner than anyone expected (for the record, everyone expected that to happen sometime during 2027).

So, here’s a billion-dollar question: With the majority of your traffic no longer human and the ad-and-subscription model it funded no longer working – how do you fill the gap? Cloudflare suggests the answer: build the “toll booth.”

What is the Cloudflare “toll booth”?

It all started in 2025, when Cloudflare launched Pay Per Crawl, enabling site owners to set a per-crawler access rule: Allow the bot in for free, Charge it a flat price, or Block it outright. A charged crawler is served on payment, or handed an HTTP 402 “Payment Required” if it declines, with Cloudflare running the booth as merchant of record.

The twist: Just a year later, Cloudflare declared that model insufficient – so, they reimagined it. The newly-dubbed “Pay Per Use” model pays the publisher when their content is actually used in an AI answer, not merely crawled, thereby effectively moving the toll from the fetch to the outcome. 

The reasoning: Crawling is a poor proxy for value. There’s actually a case to be made here, since a page might be fetched once and cited thousands of times – or it can be fetched thousands of times and cited never.

Cloudflare pushed this idea even further, extending the same model to any resource (e.g., pages, datasets, APIs, tools) by opening a Monetization Gateway, where AI agents pay per request over neutral rails, with no account required.

Why does Cloudflare’s Pay Per Use model matter for your AI SEO?

Think about it: Pay Per Use charges for the exact thing AI optimization strategies exist to earn – the citation. When the market starts paying for content because it appeared in an AI answer, it puts a literal dollar value on the precise KPIs AI SEO already optimizes toward: citation frequency and share of voice. The trajectory is unmistakable: 

Citation is becoming the new billable unit of the AI web – and you’re already competing for it.

There’s also a practical upside: Cloudflare’s new Attribution Business Insights dashboard promises to show which bots consume your content, where it gets cited, and how much human traffic each AI platform sends back. 

If this feature delivers, it will make AI visibility measurable in a way it never was (mostly), especially when combined with other similar tools like Google’s GSC’s new Search Generative AI performance reports and the GA4 AI Assistant Channel.

Should your brand toll or stay open?

At its core, tolling is a publisher play – it makes sense if your content is the product. However, if you are a SaaS company, an agency, or a local service provider (among others), your content is a marketing asset whose job is to get you discovered and recommended. As such, charging crawlers to access it would firewall the very visibility you’re after.

That being said, if you hold genuinely proprietary assets (e.g., original research, data, tool), the Monetization Gateway lets you charge agents for those specific resources, while keeping your marketing content open and citable. In short: toll the dataset, give away the blog.

What are the caveats surrounding Cloudflare’s Pay Per Use?

This is the first caveat: Cloudflare’s idea looks good on paper, but the model only works if AI companies adopt verified identities and paid access instead of simply routing around them. The problem is – none of them are obligated to.

The second caveat concerns the attribution. AI models may synthesize an answer from dozens of different sources, paraphrase without crediting, or simply cite no concrete source at all. This makes the whole “pay when your content delivers value” far easier to announce than to audit.

The reality, and the key takeaway, is that this is still an experiment, and not a settled market. Yes, it has real momentum – but whether it lands or not depends on too many fluid factors. Therefore, the best approach is to treat it as a direction, not a done deal.

Jordan Parkes, the CEO and founder of ZeroClick Labs, has been building and scaling digital marketing strategies since 2012, leveraging performance-driven SEO and data-based digital marketing solutions to guide the growth of hundreds of companies across the U.S. and Europe.