Hacker News Reader

Aren't /search and /Search considered two different directories when it comes to robots.txt?

Yes. RFC 3986, sections 6.2.2.2 and 6.2.3.

I think you got the wrong RFC number:

No mention of robots.txt, nor a section 6.

Sorry, the correct number was 3986.

RFC 3986 - Uniform Resource Identifier (URI): Generic Syntax