As large language models (LLMs) grow, the active scraping of publisher sites — followed by large-scale harvesting of that data — often happens without visibility or consent. Many content creators remain unsure which bots are accessing their work, where it ends up, or how it’s being used. Leveraging its global scale, Cloudflare has introduced tools to bring friction into this process, from a “block” button to halt unauthorised scraping in real time to a pay-per-crawl beta programme that explores new economic models. These measures are part of a broader, meaningful discussion about the conditions needed for a fair market to emerge — one where publishers can protect their work, set access terms, and ensure equitable value exchange with AI developers. Study tour participants will gain a rare inside look at how Cloudflare detects scraping activity, enforces policies, and works to create a healthier balance between openness, control, and sustainable content economics in the AI age.
Speakers