Hacker News Reader

This kind of hidden text seems trivial to exclude. Does the phrase being searched occur in a parseable sentence? If not, penalize the result. Also, it's trivial to deduce that the text doesn't display; that alone should make it excludable from any and all searches.

View on HN

"Does the phrase being searched occur in a parseable sentence?"

- that's just an insane thing to suggest, that would exclude much of the world's poetry, lyrics (I'm looking at you Icona Pop), code, etc etc etc. What does it even mean "to parse" in this context.

"it's trivial to deduce that the text doesn't display" - for that example it might be, but in general it's anything but trivial.

I think in some way, Google bot behaves as a headless browser (a la PhantomJS albeit their own special sauce). Their goal is to mimic an actual human user visiting a web page so even JavaScript may be getting parsed/evaluated so dynamically included text may also may get indexed.

In a similar vein, "invisible to user" text may get cross referenced with "visible to bot" text and if there's a discrepancy, that may count against your rank.

Reply on news.ycombinator.com