Hacker News Reader

Cloudflare's documentation says that Labyrinth is not based on robots.txt.

In line 1 of of the linked page: "waste the resources of AI Crawlers and other bots that don’t respect “no crawl” directives".

Does that indicate the robots.txt is how "no crawl" is indicated? robots.txt doesn't have "no crawl", it has allow and disallow.

And the misbehaved bots follow the path right into the pit and then...the Void of Infinite AI Abyss.

Nifty trick.