Hacker News Reader

You're reading a lot into what I posted. For the record, I was mostly playing devil's advocate, but I don't think the question is as ridiculous as the post I replied to seemed to think it is.

There is One True Web, of course, which is the collection of all documents on the web at any given point in time. But the interface to that web is nearly always a search engine which are all obviously presenting a list of suggestions according to some criteria. Google used to use PageRank, which was a relatively "objective" criteria in that at least it applied equally to all pages. Google now uses tons of data to make decisions about what to show me when I search, but neither you nor I have any idea what they are. Maybe they do actually use some algorithm which applies equally to all pages. Maybe they use some deep neural net, and not even they know how it actually works. Maybe they have a list of heuristics that they're evilly using to manipulate all the world's information. We'll never know without seeing the source code.

Again, I'm not tabling it as a serious proposal, or claiming that Google are doing anything evil. But given that they have the potential to essentially manipulate the sum total of the world's knowledge as seen by 99.999% of the population it doesn't seem ridiculous to consider whether we might like more insight into what they're doing.

> Google now uses tons of data to make decisions about what to show me when I search, but neither you nor I have any idea what they are.

To be fair, a lot of people at Google probably don't know either. Peter Norvig had a great talk [1] in which he discussed searching Google's code base for "naive Bayes" and found some code in 2006 with a funny comment:

> “And it was fun looking at the comments, because you’d see things like ‘well, I’m throwing in this naive Bayes now, but I’m gonna come back and fix it it up and come up with something better later.’ And the comment would be from 2006. And I think what that says is, when you have enough data, sometimes, you don’t have to be too clever about coming up with the best algorithm.”

[1] https://youtu.be/ql623nyCdKE?t=5m25s

I don't want to sound like an anti-algorithm-think-of-the-humans advocate...I love algorithms...but I think your statement:

> Google used to use PageRank, which was a relatively "objective" criteria in that at least it applied equally to all pages.

...besides being too oversimplyfing for even a simplification -- I don't think the evolution of Google could remotely be reduced as: first there was purity of math, then came the money -- but I also think that you overlook the bigger issue...No one argues that algorithms are statements based in mathematical truths. It's the decision to use an algorithm -- including the weighting of its tradeoffs -- that is decidedly biased and opinionated.

How is "a page should be evaluated by the number of pages (plus the authority of those pages) link to it" not an opinionated statement of the way information should be organized, versus the pre-Google algorithms of "If a page has a lot of mentions of a word, it must be particularly relevant to that word"?

The fact that PageRank had to be changed and modified as soon as people figured out how to create networks of backlinks should in itself be evidence that an algorithm -- and the decision to use it -- is not just objective truth in the way that 1 + 1 = 2 is.

> If someone from Vietnam searches for "Vietnam War"...should they be sent to the entry at vi.wikipedia.org because it's in Vietnamese and perhaps more relevant to the Vietnam user? Or to en.wikipedia.org's version, because besides having a shitton more inbound links, it's likely to have also many, many more eyes and editors on it? These are complicated things

I don't find this one so complicated; they're better sent to en.wikipedia.org, because that page, like their query, is written in english.

Sorry if this is already evident to you...but the differences between Wikipedia sites is not just about the language. vi.wikipedia.org is independently edited and maintained from en.wikipedia.org. The Vietnamese wikipedia page [1] is not as lengthy as the English one. And I'm not as good as reading Vietnamese as I should be, but the Vietnamese page doesn't seem to have the same content for "Media and censorship" that the English one does...among other differences. So I don't think it's as simple as "Send them to whatever language they used" (though I guess this is one case where seeing Google's source code would be enlightening, so...touché?)

[1] https://vi.wikipedia.org/wiki/Việt_Nam

I have no doubt that the English page is higher quality than the Vietnamese one. That won't matter at all unless the person searching can read English. When looking for information, there are no concerns greater than "even if I find it, can I understand it?"

Even if the Vietnamese page were better, there's no excuse for replying to an English query with a Vietnamese result.

Additionally, it could depend on whether they are using google.com or google.vn.

Personalization is not a problem. Systematic bias is a severe problem, even if it is not intentional.

The 'one true web' idea is a red herring. We absolutely need to know how google biases the results.

> Do you think that someone whose IP is located in Utah should get the same result for "good hamburgers late night"

Yes, I absolutely do. What if they want to find out what "people in general, according to Google" think and write about that? That question isn't even possible anymore, if you will.

> If someone from Vietnam searches for "Vietnam War"...should they be sent to the entry at vi.wikipedia.org because it's in Vietnamese and perhaps more relevant to the Vietnam user?

No, because "Vietnam War" is an English phrase. I doubt it's spelled that way in Vietnamese.

> of a worldview in which there is One True Web for everyone, which seems as problematic

How so? When I watch a movie, I see what everybody sees. If it's full of advertisement or laced with propaganda, I know it's that way for everybody. I think that has value in itself. Not every query is about getting what I want, sometimes I want to see "how the world is", and for that purpose Google, Facebook and others aren't helping with their being overly helpful. Others may feel different, but FWIW, I never heard anyone say that typing "ordering pizza in X" is just too much work compared to just typing "ordering pizza". Thinking such things are neat were, as far as I can tell, unilateral decisions made by these corporations. Nobody asked for it, nobody would miss it, and I think it's harmful just for destroying the idea of a "consensus reality" on the web even being possible.

Google's mission statement is to "organize the world’s information and make it universally accessible and useful." Currently, their opinion is that when a user searches for "late night hamburger place", the information most useful to that user is a late night hamburger place near the user, if there is such a place nearby with a statistically significant amount of noteworthiness, i.e. not just a random blog post from another user near me in Palo Alto that mentions how they loved eating Shake Shack in New York.

Your opinion is that what is usable to people around the world is knowing which page ranks highest according to a metric such as PageRank when it comes to "late night hamburger place". Because numerical metrics is "how the world is". There's no way you could be a casual habitant of Earth and think that that's how many other people evaluate things in this world.

But here's an example: What if a user types, "late night hamburger place near me"? How should Google interpret that? At what point does "pure" PageRank shut off and "Google as the new Yellow Pages" kick in? That in itself is an opinion not just about how information should be organized and what is useful, but about linguistics: whether or not that query is interpreted as a question or a user typing in the words he remembers from a song he likes and hoping that this blind query finds it.

> Your opinion is that what is usable to people around the world is knowing which page ranks highest according to a metric such as PageRank when it comes to "late night hamburger place". Because numerical metrics is "how the world is". There's no way you could be a casual habitant of Earth and think that that's how many other people evaluate things in this world.

Yes, and? The question was what I find better. I answered that. That others find something else better is not relevant to what I find better. I'm more than just a casual habitant of Earth and I notice people generally gobble up anything just because it's there and someone is pushing it. The success of corp X or Y means as much as the success of Hitler to me, I still have to make up my OWN mind, people are mostly useless even as individuals and doubly so in aggregate. What they like or don't like doesn't matter unless they can present a good argument for it. Yes, I'm arrogant. And I do not expect Google to agree, they "have a business to run". But that doesn't change my opinion.

Also, I said "how the world is according to Google".

> But here's an example: What if a user types, "late night hamburger place near me"? How should Google interpret that?

Where to buy product X or eat Y is below even trivial, I find it nauseating but not surprising that such stuff is constantly brought up. And yet that's the "killer application" I have to find a replacement for, with "why would anyone even think about that much less spend one second programming anything related to it" not being a valid answer, and my concerns about people seeing what others see not even worth being mentioned, much less addressed.

Don't interpret it at all, and people will have to be more specific, try out various phrases, which they will learn to do quickly. Adding "in [name of city]" to a query shouldn't be that hard.

Also, I'm tired of "information" being held hostage by what is merely about products and those who sell them. Yes, technically that's information too, but it's a very high-minded word for the lowliest of human aspirations in my books. The ones that matter when I am (indirectly) asked what I like or don't like.