Hacker News Reader

Comment by jetrink | Hacker News Reader

The info boxes sometimes contain surprising errors. I recently searched for Picasa and was informed that it was invented by Pablo Picasso in 2002! Google 'knows' Picasso died in the 1970s and it 'knows' he released a popular software program in 2002. That's an obvious contradiction requiring the simplest of rules to detect, but the system is just a dumb text extractor. (Interestingly, Google assistant gave the correct answer for 'who created Picasa?' so that system must use a different knowledge-base.)

I, being compulsively helpful, reported the error and it was quickly fixed. Maybe I'm part of the problem.

View on HN

A few days ago, a person in an IRC channel I'm in was convinced AMD GPUs have CUDA cores, because they put "amd 5500rx cuda cores" into Google, saw the infobox entry that said "How many CUDA cores does a Rx 5700 have? 2,560 CUDA cores" and stubbornly refused to accept Google could be wrong.

Amusingly that same infobox then has a table that states "2,560 stream processors" in the "Radeon RX 5700 XT" column. So one half of the ML parsed the web page correctly, presumably into some internal non-ambiguous representation, but then the other half of the ML interpreted it incorrectly anyway. Also, the "5500rx" (search query), "Rx 5700" (infobox title) and "RX 5700 XT" (infobox table header) are three different cards with different numbers of stream processors.

And people wonder why ML gets a bad rep.

Indeed, especially when AI/ML decide to ban your account with no recourse and no explanation ...

It starts to get pretty circular when you consider that GPUs can be used for ML.

It's so strange, that infobox has never given me an accurate answer or an accurate question! What I mean by that is, it will find some snippet that sort of looks like it answers the question, but it's nearly always misleading. It's either lacking context, creating a question that doesn't make sense or just plain wrong. I avoid the thing like the plague and look for real answers in the actual search results (which are often the same pages, but at least provide context as to what question they are answering and how)

Oh no this is worrisome, I hadn't thought to take the box data as a "maybe correct"... given that I am often googling something I am unsure about.

ML is not at fault - it's probably doing what it's trained to do. It's the marketing that makes people expect that if ML blackbox managed to do a couple of parlor tricks like distinguishing a cat from a banana and faking something resembling a human-written text, if you don't look too closely, then it's almost as good as a human brain. It's not. It's pretty much as dumb as a refrigerator (a very complex one, with many many buttons, but still a refrigerator). So you shouldn't expect too much from it, and then there's no reason to give it bad rep.

I think it's valuable to shift the focus from the information to the presentation of information. The information isn't controlled by Google, but the presentation absolutely is, and as mentioned elsewhere, makes the ML look authoritative, and seems to be implying an attribution to (if not outright falsely sourcing) Wikipedia, contradictions and all.

The tech is pretty neat and all, but this is a blatant misuse.

In this case, we also several hints that the guy is dead:

- “Hristo Bogdanov Georgiev […] was a Bulgarian rapist and serial killer”

- “Cause of Death: Execution by hanging”

- “Died: August 28, 1980 (aged 23-24)”

Yet we also have

- Born: 1956 (age 65 years)

Also, if they don’t exactly know at what age he died, how do they know he would be 65 now, and not 64?

I think Google can do better, but not at the scale they need to cover all bases in Google search.

Interesting enough it tells me that he was executed by firing squad and not hanged. So do the screenshots of the source. "Cause of death: Execution by shooting"

If you copied that correctly then that would imply that these results are personalized?

The article only has screen shots, so I had to type it, and probably made a mistake.

It's obvious for you, a human. There's no human curating those results (it's completely infeasible at Google's scales) and their graph building algorithms are extremely dumb. They are proprietary, but all of them are, so I'm sure Google's are too. The rules that could be added are obvious for each case, but there are trillions of possible cases, and there's no feasible way to create rules for even a small percent of them. So what you get is a giant mess of data that kinda sorta works most of the time, and spectacularly fails on any exception or complicated scenario. That's what we pay for doing such a massive information processing with such a relatively small amount of resources.

I’d push back on the idea that human curation is infeasable at Google’s scales. It might be more costly, but remember that Wikipedia did roughly the same task (accurate-enough text + image descriptions for every reasonable topic) with only volunteer time. There are under 10 million wiki pages, and nobody would claim English Wikipedia has any major uncovered areas. For a large-but-doable investment of, say, a billion USD, Google could spend $100 worth of human curation per Wikipedia topic. That represents a few minutes time for a skilled expert or a few hours time for an unskilled reviewer. Once you start refining your topic list, you can do even better than that.

Wikipedia relies on a vast army of unpaid volunteers. Google would never be able to buy as much time and effort, even with their trillions of dollars. That's why they are actively using Wikipedia (and Wikidata) data. And they also import errors and vandalism, and it takes much longer to fix errors than on Wiki, because see above.

> a billion USD, Google could spend $100 worth of human curation per Wikipedia topic

$100 would buy how much? Probably 30 minutes of my time, if I'm feeling very generous. Actually, probably 10 mins if it's Google, because I'm not giving Google any discounts, if I'm to work for the Evil Empire, at least I want to get rich from it! That's not counting training costs, transaction costs, legal compliance and HR benefits costs, etc. etc. So, how much work I'd be able to do with one-time investment of 10 minutes? I don't think too much, even for the topic I'm an expert in. Maybe I'll be able to notice and fix one error, once.

And then, the data changes all the time. People do new things, people change jobs, people change names, people are born, people die. You have to run very fast to just stay in the same place. And then you have 200+ world languages (surprise, not everybody speaks English!) - Wikipedia actually has 300+ but let's drop the most exotic ones.

So a billion dollars wouldn't get you as much as you'd think. You probably need to bump it by couple of orders of magnitude. Which gives you an appreciation of how much value people are actually willing to donate completely free, if motivated correctly. Unfortunately, there's no way Google could have it - except through an intermediary like Wikipedia.

> $100 would buy how much? Probably 30 minutes of my time, if I'm feeling very generous.

You're not the only one who can do the job, of course.

I searched for what year Ben Franlkin freed his slaves and was told 1863.

Hell of a feat, since he died in 1790.

Reply on news.ycombinator.com