Hacker News Reader

Cloudflare is also great at playing both sides, and they're trying pretty hard to push for pay-to-crawl because they'll probably get a 30% cut along the way.

View on HN

Pay-to-crawl works better and draws in more customers if their detection is better, doesn't it?

And I only stand to gain from pay-to-crawl, so I don't really mind that play.

That’s definitely pretty annoying, it’s almost like the walled gardens scenario. I wish this could be more democratized.

Reply on news.ycombinator.com