Slovenčina 
Čeština 
Shqiptare 
Österreich 
België 
български 
Bosanski 
Українська 
Eesti 
Македонски 
Türkçe 
Deutsch 
Dansk 
Español 
Français 
Suomi 
Ελληνικά 
Hrvatski 
Magyar 
Schweiz 
Italiano 
Lietuvių 
Latviešu 
Nederlands 
Norsk 
Polski 
Português 
Română 
Srpski 
Slovenščina 
Svenska 
The Problem Nobody Talks About
Here’s the thing – if you run a website, you’ve basically had two choices. Either let AI systems waltz in and take whatever they want for free, or block them completely and miss out on potential traffic.
Meanwhile, Google and OpenAI are cutting million-dollar deals with major news outlets while smaller creators get nothing. Their articles and blog posts are training AI models worth billions, and they don’t see a penny.
After talking to hundreds of publishers, Cloudflare figured out what everyone really wanted: let the AI bots in, but get paid for it.
Source: blog.cloudflare.com
How It Actually Works
Cloudflare dusted off an old web standard – HTTP code 402, which means “payment required“ – and built a whole system around it. When an AI crawler hits your site, you’ve got three options:
- let them in for free
- charge them your set price
- block them entirely
The technical stuff involves some fancy cryptography to make sure bots can’t fake their identity. But the basic idea is simple: no payment, no content.
Two Ways to Get Paid
Publishers can set this up however they want. Maybe you charge €0.01 per page, or €0.10, or whatever makes sense for your content.
The system works two ways. Either the bot asks for your content first, gets told the price, then decides whether to pay. Or the bot says upfront, “I’ll pay up to $0.05 for this,” and gets instant access if your price is lower.
If your price is higher than what they’re willing to pay, they get a “payment required” message with your actual price.
Source: blog.cloudflare.com
The Money Side
Cloudflare handles all the payment processing. They track when bots pay for content, charge the AI companies, and send the money to publishers. No need to set up your own payment systems or negotiate individual deals.
The whole thing plugs into whatever security you already have running on your site.
What’s Next
Right now this is in private testing. Cloudflare wants to hear from both sides – AI companies willing to pay for content and publishers who want to start charging.
They’re already thinking about upgrades like dynamic pricing based on demand, different rates for different types of content, and automated marketplaces where bots bid for access to information.
Why This Matters
This could actually change how the internet works. Right now, AI companies get most of the value from content while creators get nothing. Pay per Crawl flips that around.
As AI systems get smarter and more independent, having a standard way to buy and sell information could be crucial for keeping the web full of good content. If creators can’t make money from their work, they’ll stop creating.
Whether this experiment succeeds might determine if the future internet rewards the people who actually make stuff, or if it stays the current system where AI companies get rich off everyone else’s work for free.
