Cloudflare rolls out feature for blocking AI companies' web scrapers

I use Perplexity and it says its process is to break down a request into a series of web searches that it conducts at central servers in real time basically at my request. It then reviews the pages for relevant information and provides a summary of sorts.

I can imagine this might result in Perplexity having a bigger visibility to web site owners, because it's contstantly doing bursts of searches from a central server that aren't labeled like a web scraper. But, that's exactly how it responds in real time to my user inputs and requests.

Cloudflare's service appears to target actual web scraping instead of real time searches, so I'm not sure they actually hit the nail on the head in mentioning Perplexity in relation to their service.

See https://news.ycombinator.com/item?id=40865627 for cloudflare's blogpost.

What does the end game look like?

Browser automation tools imply that we can battle this forever. Much like with copyright and other forms of DRM/anti-cheating technology.

I don't believe there exists purpose in chasing this rabbit beyond making customers and investors believe they have a chance of actually catching it.

It's probably going to be like a firewall or antivirus or "endpoint security" market. Protection from AI intelligence as-a-service.