" A search engine doesn't discriminate in what content it indexes."
That is incorrect.
http://en.wikipedia.org/wiki/Robots_exclusion_standard
Tubetrooper certainly discriminates.
That is incorrect.
http://en.wikipedia.org/wiki/Robots_exclusion_standard
Tubetrooper certainly discriminates.
Search engines can certainly specialize, like in your (unnecessary) porn site example. But if they specialize in providing links to copyright-infringing material, make profit with advertisements, they can get in just as much trouble.
The OP made the comparison with search engines. How come they can have copyright-infringing links and not get into trouble? It is because they index all information available to them, without discrimination. And then they still have to respond to DMCA claims.
If a search engine specialized in engineering parts, it wouldn't need to host links to copyright-infringing material, so most specialized search engines are a non-issue.