Google’s Web Search Quality. A picture is worth a 1000 words | Hacker News Reader

Google’s Web Search Quality. A picture is worth a 1000 words

jakenbake.com

281 pointsstaunch94 comments

Open articleView on HN

The article shows BING & GOOG'S search results for "the film where no new babies are born"

BING returns gibberish results, GOOG returns an IMDB link to "Children of Men", the perfect answer.

But one search out of billions is easy to gimmick, what are the results with a slightly different search string:

"the movie where no babies can be born anymore"

http://www.bing.com/search?q=the+movie+where+no+babies+can+b...

https://www.google.com/#sclient=psy-ab&hl=en&safe=of...

Bing's search is more accurate in the above example

Not "knock it out of the park" accurate but GOOG returns the complete wrong answer

How many of these could you find if you had the time in your day?

That said, as a guy with a bunch of MSFT stock I can't remember the last time I used BING

https://duckduckgo.com/?q=Movie+no+babies+born+anymore :)

Ah, DDG still refusing to load any results for me. That website really doesn't want me to use it ...

Please email me -- would love to sort it out.

Doesn't DDG use Bing as it's primary search provider?

http://help.duckduckgo.com/customer/portal/articles/216399-s...

Smallest grammatical change to get the correct result: https://www.google.com/search?q=the+movie+where+babies+cant+...

"the movie where no babies can be born anymore" ->

"the movie where babies can't be born anymore"

There is definitely some room for improvement here, I'll submit this internally.

Also, compare these:

"movie where no babies are born anymore"

"movie where no babies are born"

ddg still fairly bad with that

http://duckduckgo.com/?q=the+movie+where+babies+cant+be+born...

Why does the picture show Bing in Beta? Is there somewhere where it is still in beta? Google clearly has it's new (IMO uglier) UI.

I'm in Argentina. Some months ago a Microsoft guy told the me Beta thing is because Bing has not yet been localized to Spanish. I don't know if this is currently the case, but I definitely still see "Beta" in the logo in all my searches.

Yeah. A Picture may be worth 1000 words, but 1000 words isn't enough to tell a full story.

I don't know, Knock manages pretty well with seventeen.

https://en.wikipedia.org/wiki/Knock_%28short_story%29

Bing wasn't gibberish. Bing was wrong consistently in the same way. Just a subtle point that the problem is doable. I'd be interested in broader comparisons.

http://duckduckgo.com/?q=a+film+where+no+new+babies+are+born

http://duckduckgo.com/?q=the+movie+where+no+babies+can+be+bo...

ddg is fairly bad accuracy

One of my pain points with Google is that the page is now bloated beyond belief. At one time, I'd hit the "home" button (G was my home page), the page would pop up instantly and I'd start typing my search.

Now, with "instant search", "google plus" and all that bloat, the page pops up, but when I start typing, nothing happens (as it's still loading the gobs of JavaScript); and then it'll miss the first few characters and then start pulling in random search results based on the last few characters I typed, while trying to load previews of pages. In all this confusion, the time taken for me to enter a search term and get results has gone from, say, 3-4 seconds earlier, to 7-10 seconds now. I know it doesn't sound much, but this is a company where Marissa used to count the individual characters of the homepage. Now that she's not in charge, the page has ballooned like Kirstie Alley.

Disable instant search.

Really. Are you on a particularly challenged connection? I just tried exactly that test and could barely get anything typed at all before the page display was finished. And it certainly didn't miss any characters. Obviously there is a lot more content on the page now than there was a few years ago, but AFAICT they've hidden it exceedingly well.

Honestly, the above seems like pretty bald hyperbole to me.

You may find this hard to believe but, there are places that exist that are not America.

We like to call this "The rest of the world". It's nice and there are many friendly people.

Now let's take an example of a place in the rest of the world:

Perth, Australia.

What's the flight time of a packet from Perth to Mountain View? 300ms -400ms? Taken each way that's almost a second.

Hyperbole? not at all.

You may now continue living in total oblivion to the world around you.

Also you say they've hidden the extra content. The same extra content which was just more ads. You're actually making the contention that google was cleverly hiding all the ads from you?

No it wasn't. It was cleverly pushing real results below the fold where they will stop distracting users from the ads.

see this: http://www.seobook.com/excuse-me-where-did-googles-organic-s...

This is only heading in one direction - "Pay up for adwords or you're on page two, buddy".

You do know that your Google searches don't go to Mountain View, right? Google has datacenters all over the world. I think you know better, and I'd love to see some actual numbers from you.

I disagree with most of the OP's points, but...

You do realize that Google doesn't have a datacenter in Australia, right?

Our searches go to California, so I think the OP's point about the round trip time to Mountain View are reasonable.

I'm not in Perth (which - to be fair to Google - is the most isolated city in the world), but from Adelaide my timings look like this:

GET http://www.google.com.au/s?hl=en...

Waiting: 205ms

Receiving: 218ms

Total: 423ms

Having said that, I usually get results before I can press Enter.

You are free to use a non-American search provider that is more receptive to your friendly non-American needs. I suggest Baidu.

I have this happening regardless of connection speed (right now speedtest.net shows 6.55 Mbps). It's not only about speed though, you have 15 tabs open and google.com loads slower. Before it wasn't a problem, but now there's that brief moment sometimes when the page is loaded, but doesn't work yet.

Even worse with Instant Search on, and you can't disable it in your account—it seems to be saved in cookies independently, so you'd have to regularly visit Search settings.

It was quite distracting when I used google.com (now, I usually search via Chrome's omnibar).

If you were really obsessed about speed, you'd use the omnibox.

Completely agree, its sad what the G homepage has become since MM was moved. Loading, waiting, starting to type, missing the first characters, loading previews for the last few chars, trying to delete the incomplete input, etc. happens to me all the time, quite annoying. Main reason I mostly switched to DDG by now. Not concerned about privacy, only hate GOOG's slow loading/render.

I get this on my phone (HTC Incredible on the Verizon network). Google's instant search is nearly unusable as it causes the phone to grind to a crawl, miss keystrokes, bounce the page around and generally act weird.

Add a new search engine to Chrome, name it anything but "Google" and make the search string this:

{google:baseURL}search?ie={inputEncoding}&complete=0&nfpr=1&pws=0&safe=off&num=30&q=%s

It disables instant search + autocomplete, autocorrect, safe search and sets the number of results to 30.

Set it as your default search engine. When you clean your cookies out or go incognito, you won't lose your search settings.

If you want the "cached" link back, install this extension - https://chrome.google.com/webstore/detail/nhihjhedaljdlpkcpf...

I just tried this with a different movie in mind. The results are pretty interesting, but see for yourself:

http://www.google.de/search?q=the%20film%20where%20a%20house...

http://www.bing.com/search?q=the+film+where+a+house+is+flyin...

http://duckduckgo.com/?q=the+film+where+a+house+is+flying+wi...

If you get the same results as I do, you should find the correct answer as duckducks first answer and bings 4th one. Google's first page of search results has no reference to the correct answer (though in the picture search the first one picture is from the movie).

I use google day in day out. Maybe I should overcome that habit :-)

My first result in Google for that query is: "National Geographic real-life floating house: Pixar's Up! can be ..."

In any case, I think I have my search-fu already optimized for Google: I'd search "animation film flying house" instead, which results in:

Google → 1st result, Wikipedia page for Up!

DDG → 3rd result, reference to Up! in the title, although not a page about the movie specifically.

Bing → Up! nowhere to be see on the first page.

I see several relevant results in Bing including the wikipedia entry for Up as the fourth result.

Well, it definitively doesn't here: http://i.imgur.com/zaiCy.png (I joined two screenshots in Gimp, that's why the scrollbar is strange)

I've been curious what accounts for variation between ddg serps and bing serps for the same terms aside from the zero click and ads. Does bing personalization account for the differences, or do partners like ddg have the opportunity to influence weightings in whats returned? Is Gabriel explicitly searching wikipedia independently?

doing an English language search on google.de is kinda unfair. Also note that the hl param on google is set to german. Of course you get crappy results.

Bing will try to autocorrect incorrectly. If you stick a + before the query, it'll return the right result:

http://www.bing.com/search?q=%2bthe+film+where+a+house+is+fl...

In fact, as far as I can tell, Duck Duck Go's first result is always identical to Bing's (if you turn off Bing's phrase correction). I imagine this is because DDG uses Bing and always makes it their first result.

Google has many cute gimmicks built in, no doubt. But honestly I've been frustrated at Google's results as of late. Google simply has been SEO'd to death. Plus, uncommon phrases are completely swamped by more common similar ones. Search is just waiting to be blown wide open again, at least in some specific cases.

I've been having the same experience. It's all too clear people are gaming Google to get on the front page. A few years ago, I never had to really search for stuff I wanted on Google. Now, I really have to weed out a lot of useless results to find what I want.

Would love to see some example queries to debug if you guys have them handy. (You can check your search history https://www.google.com/history/ to jog your memory if you have it turned on.)

If this a genuine request for feedback, I will happily share my experience with the query "learn chinese".

Result #1 (-tools.com) is part of an interlinked series of clone sites. Result #2 (clear-.com) has zero original content. Result #3 (-wen.com) is decent. Result #4 (-mm.com) is low-value. Result #5 (instant-.com) is a spam site with no content. Result #6 (rosetta-) is OK given the strength of the brand but is not the most obvious choice. Results #7 and #8 are the exact same page. Result #9 is a spam site with no content. The in-lined videos are meaningless and low quality.

Things get worse on page two, with almost all of the sites that rank, ranking because they spam. Compare the results with what you'll get searching the Apple podcast directory (which lists stuff people actually use and cuts spam by actually requiring content) and you'll see the quality of the offerings and the user experience is literally night and day. None of the highly-rated stuff there shows up in Google.

So assuming the results should be the leading free and or paid services or resources for learning Chinese, Google has a total fail, with not a single relevant result on its first page, and only a single hit on page two. If I'm looking for sites to spam me or put me into a high-pressure sales funnel, I basically hit bingo though.

Thanks!

My only real beef with Google search at the moment is that it seems to give too much weight to "current events." If there is some big piece of news (like the recent school shooting), it funnels you towards that result. I was looking for some statistics on suburban schools, and a search for 'suburban schools' returned 9 results about the shooting that occurred the day before, and result from ehow.com

It would be pretty cool if we could have knobs that would control the weighting of different factors - news, wiki, help websites, etc.

I guess I don't really know how it works though, so maybe that's not very feasible (or maybe it'd be too easy to game?).

With sort of the opposite result for tech searches. It's very common to get links to blog and forum posts from several years ago when it's far more likely that information posted in the last year would be more helpful.

You can limit the time span of your search in the sidebar on Google. I'll often do that when I'm searching for something technical.

!!!! Awesome! I can't believe I never noticed that. This 100% solves this specific case for me, since I can just tailor the time range to before the current event occurred. Thanks so much :)

Excellent for "best way to install X" posts when X is fast-moving :)

You can restrict your search by date range. So you can exclude everything from the last two weeks, or the current year.

https://www.google.com/search?q=middle+school+statistics&...

If you encounter these, please submit it to http://www.google.com/quality_form

Search quality feedback is key to improving the search engine.

> Google simply has been SEO'd to death.

I feel like they have gotten a lot better over the last few months. I guess it depends what subjects you search for. For a while I thought "personal filter" was a good idea but lately I don't feel a need.

[the film where no new babies are born] → [no babies movie]. The rest of the query is superfluous or implied.

http://google.com/#q=no%20babies%20movie finds it.

"no babies" does not appear on the IMDb page itself, but I expect it appears near many links to the IMDb page.

Edit: rewriting queries is a necessary step to do question-answering with a search engine. Query rewriting + some sort of knowledge graph is a large part of the Watson recipe; the graph is comparatively sparse but simplified queries can be entry points into that graph.

Google does a stellar job correlating click through and click back actions + times to determine what people were most likely looking for with each query, and then raising or lowering results accordingly (Panda amplified this metric in their scoring system). I suspect they stomp Bing in that department.

Bing too shows results based on user interactions (collected via a toolbar or via IE's "suggested sites" feature):

http://googlesystem.blogspot.com/2011/02/google-results-one-...

The problem is when the cleverness doesn't work. Then it's clippy time.

I desperately want a “don't be smart button”. Searching for code is pointless, even on quotes and verbatim.

http://www.google.com/search?q=%22this%3Aempty%22

http://support.google.com/websearch/bin/answer.py?hl=en&...

That's not cleverness gone bad. You've never been able to search with a colon on regular google. I miss code search too.

about a year ago this came up

https://www.google.com/search?q=that+hipster+song+with+whist...

http://i.imgur.com/63uJd.png

the keywords in description were added after this search got popular!

This thread is filled with great anecdotes. If you do encounter something where Google is not returning the best result for your query, fill out the Search Quality Feedback form: http://www.google.com/quality_form

This will help the team improve the search engine.

I'd be interested in how those results were obtained, because when I search google with that exact same query, the results look pretty much like bing's results.

Looks like Google finds ways to keep getting better. It's an arms race... Geogle wants to be so much better than everyone else that people won't mind ads thrown in. That's ambitious. They also have to stay ahead o SEOs who compete with their ads.

Google is getting better for "general searches", but I have gotten consistently worse results over time for obscure things, like searching for code or searches for things where the results would overlap with something popular but unrelated.

This makes a lot of sense. Their search results can't self teach for obscure things.

Similar queries give incredible accurate results ("movie with Brad Pitt and Edward Norton","movie where james stewart is afraid of heights") , but it seems to work just with movies...

I got spottier results trying with music and games related queries. Google probably picks up the words "movie" or "film", and than search the rest of the query restricting the first few results to movie-related websites.

Still, it's clearly not a "Number of French military victories" kind of gimmick, search is definitely going in that direction in the following years.

great example! another improvement from google - recognize search queries typed in wrong keyboard layout, ie: https://www.google.com/search?hl=en&q=%D1%81%D1%80%D1%88...

Adding some very popular words to a query really messes up the results of all three.

I tried combinations of the movie with (four) guys in white clothing/suits. It wasn't until I searched for 'the movie with guys dressed in white' that any relevant results were returned.

http://duckduckgo.com/?q=the+movie+with+guys+dressed+in+whit...

http://www.bing.com/search?q=the+movie+with+guys+dressed+in+...

http://www.google.nl/search?q=the+movie+with+guys+dressed+in...

Arguably it's Google > Duck > Bing

Google web search's secret: it's not searching what you tell it to search.

- suggest spelling corrections and alternative spellings - personalize your search by using information such as sites you’ve visited before - include synonyms of your search terms to find related results - find results that match similar terms to those in your query - search for words with the same stem, like "running" when you search for [ run ]

http://support.google.com/websearch/bin/answer.py?hl=en&...

Also https://en.wikipedia.org/wiki/Google_bomb

If you search for chrome, Google thinks you probably want to do the following things more than you want to download Chrome from Google: buy a Chromebook, use the Google Chrome Beta, buy a Google Chrome messenger bag, read about Google Chrome on Wikipedia, download Google Chrome from download.com, download ad block for Chrome, read news about Chrome, download Angry Birds for Chrome

If you search Bing for chrome, the first result will allow you to download Chrome from Google. :)

This is probably why: https://plus.google.com/u/0/109412257237874861202/posts/NAWu...

the webspam team has taken manual action to demote www.google.com/chrome for at least 60 days. After that, someone on the Chrome side can submit a reconsideration request documenting their clean-up just like any other company would.

That could explain this: Google Chrome loses market share for second consecutive month.

http://www.maximumpc.com/article/news/google_chrome_coughs_m...

I just did this search yesterday to install Chrome on a new machine, and the chrome browser website was actually the first result. But it was a "paid" result, shaded as those things are, and I mentally tuned it out. I looked down the page in frustration that the browser site wasn't coming up and only then noticed the "paid" link. Which probably says something about the effectiveness of those paid links.

I just tried searching for 'chrome', and an ad appeared at the top of the page leading straight to the Google Chrome download page. The first real search result was a link to that comic strip about Chrome's development.

Counter example query that returns the same first hit: "movie where they use math to find patterns in the torah"

http://www.google.ca/search?q=movie+where+they+use+math+to+f...

http://www.bing.com/search?q=movie+where+they+use+math+to+fi...

The success with negation-based queries displayed in the example is thoroughly explained in The Structured Search Engine -- http://www.youtube.com/watch?v=5lCSDOuqv1A

TL;DR - Google has added phrase chunking, weights positive/negative words in phrases, and allows negative phrases to modify the search impact of the affected phrases.

No one is interested in why bing is suggesting word "baby" instead of babies? I submitted the same query and it does not suggest that anymore.

Paradoxically this is also the reason why the biggest advertiser's search engine sucks more and more. This fuzzy search where he looks for all kind of synonyms instead of what he's asked means it often returns no useful results and misses the target by a solar system or two.

Looking for a film on the web = imdb no babies are born = no procreation

So let's search for "imdb no procreation": http://lmgtfy.com/?q=imdb+no+procreation and http://www.bing.com/search?q=imdb+no+procreation

Now let's turn off this fuzzy synonymous search, go on the results page for "the film where no more babies are born", click on "more search tools" in the sidebar and click "verbatim". Not only the actual results doesn't show up anymore but we now see this very story referenced several times further skewing the results towards confirmation bias: https://en.wikipedia.org/wiki/Google_bomb

I switched from metacrawler to google for my web searching in 1998, I can testify that it's getting worse and worse at providing relevant results for a while now and at the same time it gets better at tracking users and raising privacy concerns, censoring results, adding clutter, spam and ads, silently removing useful features. I've switched to duckduckgo in 2010 and rarely go back to google anymore.

http://www.google.com/search?q=that+movie+where+the+guy+miss...

http://www.google.com/search?q=that+movie+where+the+guy+gets...

http://www.google.com/search?q=that+movie+where+the+guy+solv...

http://www.google.com/search?q=the+movie+where+that+guy+solv...

http://www.google.com/search?q=that+first+movie+with+the+din...

http://www.google.com/search?q=the+movie+with+the+space+miss...

Isn't that second one "every movie with a space mission"?

This says nothing about search quality. No single query can. The fact that I've completely stopped using Google search for about 3 months now and don't miss it at all is probably more telling. Five years ago that wouldn't have been possible, but today Bing is as good (sometimes worse, sometimes better).

You appear to be arguing that one anecdotal query "says nothing" but one anecdotal user experience says something. What is it about your anecdote that makes it meaningful in a discussion about services with hundreds of millions of users?

There is no point of linking to google search results because it is personalized. Everyone sees different pages.

use &pws=0

Doesn't help. https://www.google.com/search?q=internet&pws=0 returns only sites in local language. Zero English.

And nobody else uses &pws=0. Real Google isn't "&pws=0" Google.

&hl=all

&filter=0 (if you like)

yes there is no "real google result" but the impact of personalization (as long as you are not self spammed with google plus your world) is in generally overrated.

It's not personalized to the point where you get completely different results for each person. The only thing is the order might be slightly different, and you might get local results now.

This post illustrates the reason I cant seem to shake my Google addiction.

Whatev. So one search result (or even 100) are better on Google. Bing could be better at other things. The problem is no one uses Bing, so it's hard to know. http://imgur.com/peK58

Very bad example because Google trends to favor large and authority websites. Try a query where the answer is on a small (but very well informed) site.

Thanks you for the downvote. But if you try this query: "film no baby", you get the same result again, just because IMDB if a first choice for Google for any query related to "film". The query "film end of baby" shows different results. This downplays the presence of any intelligence in Google's results.

you mean baby born with two heads isn't it?

Microsoft's entire business is based around the idea that inferior products don't matter as long as you have the monopoly position, backed by business and sales channels that enable them to shove half-baked products and poor support down customers throats.

Microsoft is long past the point where they will continue to get away with shipping turds. I think they've lost the entire mobile computing market forever. It doesn't matter how much they polish Windows 8 because consumers and business alike are finally starting to see the light from the permanent hangover that Microsoft has cast upon the entire IT industry since the mid-1990's.

Yea Office 2007 and Windows 7 are turds compared to the competition and we are better off with Google having 100% because only they have access to the most keyword click data. I am surprised you didn't call them M$.

You haven't really given me much to dispute, but I digress. Neither Office or Windows 7 is geared toward mobile computing, which for all intensive purposes is the future of computing and which Microsoft is at an enormous disadvantage. Microsoft is currently dominant in every area of computing that is declining in every metric worth measuring. Nearly every way Microsoft makes money has become stagnant largely due to the dominance of Microsoft in these areas and the lack of innovation required to continue making money. Microsoft is in much graver danger than they let on. The entire MS stack has been abandoned by the next generation of innovators. Find me an 18 year old MS Visual Basic programmer and I'll show you a minimum wage earner for the rest of his career.

Microsoft is a dying company, and the sooner young people abandon this company and all its technologies the better off they are.

> Neither Office or Windows 7 is geared toward mobile computing, which for all intensive purposes is the future of computing and which Microsoft is at an enormous disadvantage.

Yes, too bad that MS isn't making something like Windows 8 or Office 15 that are geared for mobile and touch friendly launching in like six months or so.

>The entire MS stack has been abandoned by the next generation of innovators. Find me an 18 year old MS Visual Basic programmer and I'll show you a minimum wage earner for the rest of his career.

I just searched for 'Visual Basic' on indeed.com and came up with the following results:

Salary Estimate $20,000+ (35031) $40,000+ (16019) $60,000+ (9512) $80,000+ (4628) $100,000+ (2151)

Minimum wage is 12K/year.

Do a search for C#:

$50,000+ (28977) $70,000+ (18380) $90,000+ (8590) $110,000+ (3348) $130,000+ (1337)

These are counting only the open jobs(many of which are going unfilled).

>The entire MS stack has been abandoned by the next generation of innovators.

You mean by people like StackOverflow?

>Microsoft is a dying company, and the sooner young people abandon this company and all its technologies the better off they are.

Yes, people should do that. It will make hiring for all the above open positions easier, since it will weed out people who are easily misled by people like you and who live in a well and shout la-la-la to any voice of reason.

  Windows 8 or Office 15 that are geared for mobile and 
  touch friendly launching in like six months or so.

5 years behind the competition. Windows is so far behind iOS it's embarassing.

  I just searched for 'Visual Basic'... Salary Estimate

What you're not able to do a bing search for is how long-term these high-paying Visual Basic jobs are going to last. There's no future in Visual Basic because it's not for building apps in a mobile computing paradigm. It's an old-school desktop GUI paradigm, the paradigm that is quickly being replaced with mobile computing... where Microsoft is at an enormous disadvantage. Nobody is building mobile apps using Visual Basic.

  >The entire MS stack has been abandoned by the next generation of innovators.
  You mean by people like StackOverflow?

Out of every 100 Silicon Valley startups, I think you'll find about 1% or less are using any Microsoft technologies. You conveniently provided one of those 1%.

"for all intensive purposes"

If you're going to talk in cliches, try "for all intents and purposes".

This thread is quickly filling up with anecdotes and such which is fine, but I thought I'd throw out a tip. Do not be afraid to play with the date range for limiting or further specifying your search. I probably use it 2-3 times a week, it's very nice.

Reply on news.ycombinator.com