TLDR · PROGRAMMING, DESIGN & DATA SCIENCE

CLOUDFLARE'S NEW MARKETPLACE WILL LET WEBSITES CHARGE AI BOTS FOR SCRAPING

techcrunch.com5 min read

Summary from the TLDR newsletter

Cloudflare plans to launch a marketplace in the next year that will allow website owners to sell access to AI model providers to scrape their site's content. It is part of the company's plan to give publishers greater control over how and when AI bots scrape their websites. The company launched free observability tools for customers on Monday as part of this plan, giving website owners a dashboard to view analytics on why, when, and how often AI models are crawling their sites for information. It is also letting customers block AI bots from their sites with a click of a button.