Hacker News Reader

Comment by tshaddox | Hacker News Reader

> Some people will really not know (think kids for instance, but not only)

It seems very improbable that a kid would ask that question without having any preconceived notion whatsoever of a person with that name famously stepping foot on an extraterrestrial body. It's overwhelmingly likely that they have the famous 1969 event in mind, and that's why Google's response is appropriate.

Of course I'm not disputing that it wouldn't be even better for the Google response to explicitly correct the mistake in the query. I'm only disputing the fact that it's "broken." If you were asking this question of a human as some sort of test (but the human didn't know they were being tested), it would clearly be a "gotcha," and you could likely "fool" highly educated people who know full well that it was the Moon and not Mars.

View on HN

> It seems very improbable

This is I think the critical point.

It’s one of the core assumption that is correct for us to have in our day to day interactions with other real people, and we also are able to adjust the probability levels looking at the person or the context, and follow up depending on the reaction to our answer.

None of that applies to Google Search [0], and we are fed with “most probable” results without qualifiers, little to no sensible adjustments, and very little control or opt out options.

Billions of people use Search every day, at these scales what is “improbable” actually happens millions of times, and I feel too many people are willing to throw the odd ones out under the bus, even if the current situation isn’t perfect either for the 90 part of the 10/90 split.

[0] search personalization based on logged in profile could be used, but in practice they only apply that to very crude adjustments like country, language and frequent searches.

There is something to be said about how failure cases are handled even for systems that are almost always (say, 99.9%) successful. This is particularly true for systems where failure cases cause harm to people. A common example might be assumptions about people's names that are implemented in some form on a government website. Perhaps it is the case that 99.9% of people in this jurisdiction do have a first name and a last name each of which are representable in Unicode with a character count less than 50. But that's still 0.1% of people who now can't get a driver's license, and that's unacceptable.

However, in this particular case, I think the very low probability of someone genuinely asking when Neil Armstrong set foot on Mars, combined with the very low probability of any measurable harm being done to those people by Google's response, makes me conclude that this is reasonable expected behavior and not something I would call "broken."

You are bringing a very good point.

On the Google side, I think it’s an issue that has more consequences than just the moon landing.

For instance, for me “When is the francis election” (where I would have men “french or france instead of francis”) gives me a big and bold “Mar 13” with smaller below “Anniversary of the election of Pope Francis Observances” and an long anniversary Year/Week/Date table taking 80% of the widget display.

And as with the other examples, there is nothing showing the word approximations that has happened regarding to the original query.

There must thousands of other instances where a search result will come up with a big widget, an answer in big and bold font, except it will be completely wrong and have a direct impact on the user missing a deadline or taking the wrong action.

Of course users are supposed to know better and check the full result, but as you point out, if it’s almost always what they expected, they’ll learn to rely on it and be more complacent.

The answer Google gives to the question is wrong. End of story. Google aims to be a question-answering machine with the Info box, so giving the wrong answer is a broken state. Not sure you can really reason you way into how it's kinda-understandable-and-gotcha! to be 'not broken'.

Aside: Going to get real confusing when another Neil Armstrong born in the 2010s does set foot on Mars in the 2030s. I wonder if we will still get the first response from Google, 1969, and if you will still consider it to be 'not broken' then too.

How is this different from answering the query "2+22=?" with "4"? There's a factual question that was asked and the answer provided is objectively wrong. Whether the user meant to ask a different question is irrelevant. Either say "2+2=4" or say "24", but under no circumstances say "4" without noting that you've answered a different question than what was asked

It's different because nobody is actually going to ask this silly question in good faith. It does not matter in the slightest bit to anyone whether Google can produce a correct answer to nonsense questions that nobody is ever going to actually ask with the intention of getting a correct answer. Google also can't produce correct answers to queries written in ancient undeciphered languages. Oh well.

It also seems rather unlikely that someone would search for "2+22", but not obviously any more or less unlikely than "2+2", and building a calculator into your search engine is trivially easy compared to handling natural language queries, so it's not a useful comparison.

> Whether the user meant to ask a different question is irrelevant.

Of course it's relevant. If Google can correctly determine what question the user meant to ask, then obviously Google should provide an answer that provides value to the user instead of trolling the user with nitpicking about spelling or what have you. If you ask Google for the "capitol of India" you'll get New Delhi, even though that is the "capital" of India, and arguably the "capitol" of India is actually the Chandigarh Capitol Complex, so the result is "objectively wrong".

Google can interpret the intended meaning of arithmetic expressions like that with even higher confidence than this Neil Armstrong query. But the confidence with which Google can interpret the intended meaning of this Neil Armstrong query is well above any reasonable threshold one might propose.

I would argue that it's roughly just as clear that the query intends to refer to the Moon as it is that the query intends to refer to the Canadian Neil Armstrong who was killed in a plane crash in the Antarctic in 1994.

Reply on news.ycombinator.com