Comment by user24 | Hacker News Reader

Comment by user24 | Hacker News Reader

The problem is that Google actually aren't good enough at search yet.

They still rely on this dumb word-based approach to document retrieval. Example, if my page is about how to manage your time, and it's a really fantastic resource about that, but I don't actually mention the phrase "how to manage your time" anywhere, I won't rank for people searching for that phrase. I should, but I won't.

So I have to write my content for two audiences - humans and Google.

I don't want to do this. I'd be much happier just building great content for my human readers, and if I happen not to mention the exact keyword phrase a searcher might use, it doesn't matter, Google still knows it should put my site at the top for that phrase, because it understands that my site is about that phrase, even though I don't mention it exactly.

But Google just isn't good enough yet. Someone can set up a page which has a bunch of headers and URLs and variations of the keywords and beat me to #1, even if their content is utter rubbish for human consumption.

That's why SEO still exists. It's symptomatic of a bug in Google.

This penalty will help, hopefully, but we're still going to be in a state where we have to compromise our content to serve two masters.

Google is actually pretty good at getting the semantic meaning, but the SERPs show snippets of the content, and users only click on results that match the query.

Google could stop bolding the keyword matches in the SERPs, but that would be a disaster because they are useful for picking the best result.

My point is that this is not purely a Google bug, but also a bug in human nature. People like exact keyword matches to their queries.

> the SERPs show snippets of the content, and users only click on results that match the query

I'd love to see a source to back that up. The only data I've seen on this is various eyetracking and/or click tracking studies which suggest higher up on SERP=more clicks.

just run a ppc test with dynamic keyword insertion and look at your click through rates.

This is absolutely correct!!!

As I understand it, google is putting a lot of resources into semantic search atm which may go some way towards fixing the problems you're describing.

Yeah semantic search, if solved, would address this problem.

That's really what I was getting at. Stripped right down, Google is still just viewing documents as a bag of words[1]. I mean, they have pagerank and they will apply more weight to words in headings, and they have 6-gram indexes and synonyms and all that clever stuff, but at it's core it's still lexically centered not semantically centered.

[1] Further reading: http://en.wikipedia.org/wiki/Bag_of_words_model

Meaning Google makes no effort to score your whole page against each word in the dictionary, it only attempts to score 1- to 6-game from the page and inbound links? That's unfortunate.

No. It means that for your comment, google will store each word, each pair of words, each triplet of words and so on up to 6 word long subsets. So if someone searched for 'Google makes no effort' then your comment will come up higher than a document which includes those words in a different order.

http://en.wikipedia.org/wiki/N-gram

Is there a search engine that doesn't share the same blindspots, in your opinion?

not yet, to my knowledge. My old IR lecturer (Stephen Clark) used to think highly of Powerset, who look like they've since been acquired by Bing. And while the launch and implementation of cuil was an epic fail, it did seem to have an interesting approach, and I'd have liked to see what it could have offered.

Reply on news.ycombinator.com