Hacker News Reader

You're thinking robots.txt https://www.reddit.com/robots.txt and it's still there

Ah yes, bender was in the robots file. But we also had a funny X-header. Maybe you can find it in GitHub in the haproxy config.

Not there, but it does have "x-moose: majestic"

> User-Agent: bender

> Disallow: /my_shiny_metal_ass

This was there when I checked just now; was it removed and re-added?

Yes I saw that, but the parent comment said about it being a header ;)

The way I see it, the parent said bender is in robots.txt, and you said it's not, and then you also said x-moose was in robots.txt :)

You thought you were replying a level higher.

Lots of other websites have similar headers :)

I thought it was Slashdot that had the X-Bender header?