Hacker News Reader

Comment by OscarCunningham | Hacker News Reader

Lots of old web forums are actually not indexed. For example try finding https://conwaylife.com/forums/viewtopic.php?f=7&t=95 by searching for "Getting RLE in Golly" in Google. I get zero results. (Although of course that page might be indexed now that I've linked to it from HN. You can find a similarly old post on that site to check.)

View on HN

In my experience Microsoft Bing does much better than Google at returning search results from old forums. Google has a bunch of things just flat out missing in its search index, even things I'm sure I found by Google years ago and that still exist on the same sites.

I obviously didn't check before you posted your comment, but I checked before I posted mine and Bing found your test case with no difficulty.

In my experience Google search results have gotten less and less informative for a very long time(last 4-6 years). Seems to me Google at some point in the past decided it wasn't going to show everything, only what it meant i needed to see, monetization on the other hand has gone up.

Agreed , although I give them the benefits of doubt because modern internet can make far more web pages than Google could keep up with indexing. Especially after the Smartphone took off and Internet seems to be exploding with Content Farm.

Google used to have a "Discussions" search option that you could select, which would return results from forums that they indexed.

HN comment links have a rel=nofollow. I'm not versed in SEO, but I've assumed this means linking here won't affect indexing.

Does that make it not get indexed, or not get included in weights?

It will index those pages but it won't pass any "juice" (aka reputation).

You would use the rel="noindex" to tell Google not to index that page.

It will probably also confer at least some reputation. Or maybe not - that would explain why search results are getting worse with pretty much anything posted by lowly users will have rel="nofollow" these days.

That's correct. When I was researching why Google wasn't indexing many of our threads, I found the same thing was happening to other large forums.

Reply on news.ycombinator.com