Hacker News Reader

They'll only have to field those questions if they display the plain averages despite not sorting on them. If we've established that average rating is the wrong metric, why would you even show it to the user/merchant/whoever? Display the same metric that you sort by, and then the merchants will be perfectly happy with higher-rated items showing up higher on the list, and Big Boss will be happy that you've chosen the #1-recommended option.

I really don't think that solves the problem; you look at product X, it has 0 reviews, you give it a 5 star, now it shows 1 rating and 3.7 stars?

So you let people rate it with 5 stars, but then show it as 3.7 magic points. Magic points can be your proprietary ranking system.

You don't need to let them rate with stars and also show it as stars.

If I saw that happen, I would probably assume that the item was so popular that a bunch of other people had rated it concurrently. Unless the number of ratings were displayed simultaneously. There's a bit of a comprehension barrier in letting people know that the ordering score is not the same as the thing they're inputting, and what the relationship between them is. I don't know how to solve that problem (if I had to, I'd probably just not show the ranking numbers at all- the user already sees the sorting order, what do they need the raw numbers for?), but that's OK, 'cause I prefer to sidestep it- just don't use a graded rating scale like "5 stars". At least not unless you have a really good justification and explanation of what precisely the scale means. 5-star rating systems tend to result in large numbers of 5s and large numbers of 1s anyway, so you might as well just go with a binary like/dislike system. The fact that a ranking is a real number and your input is an "up"/"down" enumeration makes it clear that the ranking you see is not the same thing as the data that you put in. It could also be useful from a defusing confusion standpoint to make the ranking more complicated by mixing in an age factor (like HN does) or some other extra band of information. Then if a user doesn't immediately understand the relation between their rating and the final ranking, they can easily write it off as being due to some other factor that's not in their control anyway.

You can do all stars but if you say you need at least X people to rate before showing any. That's what many sites do.

The problem isn't that it's impossible for humans to understand, nor that one can't hire the best in the field to implement this solution. The problem is that the people who need to understand the outcome won't be able to understand it.

I disagree that people won't be able to understand it - I propose they can't be bothered to understand it. Regular people do not give a shit about algorithms - either the computer/software works or it doesn't. Tech geeks regularly forget this simple fact.

Most people have other things they care about and all they want is to press a button and have an answer come out.

If its instantly clear how the answer is generated, they think about it for a very short while, agree and move on with their lives.

If it's not instantly clear how the answer is generated, they assume the computer is right and move on with their lives.

> "My product has a 5-star average, but this competitor's product only has a 4.5-star average. Why the hell is my product shown further down in the list?"

'Because you have fewer votes than your competitor'

I don't think it's that hard to understand.

Right. If you look at most-helpful ratios for product reviews on Amazon, you will see this. 500/600 helpful floats far above 10/10 helpful.

the article also misses the human factor. And intent.

for amazon, is it better to show the best product first or to help sell a few of the lacking-rating ones to generate more ratings and work out the uncertainty with real world data instead of crazy math?

I don't think it is a good idea to misuse your customers for that kind of testing.