Hacker News Reader

Comment by slig | Hacker News Reader

If you try to do that on a site with Cloudflare, what happens? Do they read the zip file and try to cache the uncompressed content to serve it with the best compression algorithm for a given client, or do they cache the compressed file and serve it "as is"?

View on HN

If you're doing this through cloudflare, you'll want to add the response header

    cache-control: no-transform

so you don't bomb cloudflare when they naturally try to decompress your document, parse it, and recompress it with whatever methods the client prefers.

That being said, you can bomb cloudflare without significant issue. It's probably a huge waste of resources for them, but they correctly handle it. I've never seen cloudflare give up before the end-client does.

Cloudflare has free AI Labyrinths if your goal is to target AI. The bots follow hidden links to a maze of unrelated content, and Cloudflare uses this to identify bots. https://blog.cloudflare.com/ai-labyrinth/

Do you think Meta AI's llama 4 failed so badly cause they ended up crawling a bunch of labrynths?

Reply on news.ycombinator.com