Hacker News Reader

Comment by fsckboy | Hacker News Reader

Ratings should always be on a curve for the rater, and on a 10 point scale, with no official meaning attached to the number of stars. If you rate everything 10 stars, you just gave everything 5 stars. If you want your 10 stars to mean 10 stars, you need to rate a bell curve of other stuff too.

(you might say "but if I only select the best places, and the ratings are always confirmed, that would be downvoting them." No, it wouldn't, that's exactly the type of situation that Gaussians know how to handle. Whatever is the average rating and whatever is the excellent rating will emerge from the collective.)

if you wish to attack my plan, I'll only listen if you offer a defense of the status quo, or offer a better improvement plan.

View on HN

Counterpoint: steam reviews with their binary choice result in a better aggregate experience and remove the need for bell curves.

Binary choice leaves less room for interpretation and fuzzy ness, you either like something or you don't.

Steam has other problems with user submitted genre tags. Make a DLC that people thought under-delivered and you be may surprised to learn that you shipped a hentai game. Though that is definitely on the user’s side.

Positive negative and meh is my preferred. Some things aren’t great, but were cheap enough to justify it.

Maybe make it a really tiny box in the middle that you need to zoom in to click to encourage mostly positive or negative reviews.

How about...

    - thumb down
    - thumb up
    - two thumbs up

I've heard the arguments for switching to binary choice (netflix also did a big writeup about it), but I'd still prefer the ability to give a bit more nuance in my rating. It feels weird giving a movie that is okay and a movie that is excellent the same rating.

Rotten Tomatoes goes further to emphasize the "love it or hate it" dichotomy - they display a ripe red tomato for a movie with a rating of 60 and a splattered green tomato for a movie with a score of 59.

You might prefer binary choice on more granular criteria: Like the pacing of the overall plot, dislike the performance of the antagonist

My issue is that we’ve been trained to see things on a broken scale - 10 is good, 9 is good, 8 is probably ok, 7 is questionable and everything else is really bad.

I get it’s chicken/egg, but I feel like you can’t adjust ratings without retraining people on what ratings mean. I guess that means I think we should probably do away with the numbers and think about a more qualitative approach? Not a strong belief.

there needs to be a way to indicate something exceptionally good, and everything can't be that.

if 5 is the center of the new scale where the current grade inflation puts everything, people will not need to be retrained. Then, let them vote. You don't need to understand grading on a curve to be graded on a curve, and the grade you get is the right one.

> if you wish to attack my plan, I'll only listen if you offer a defense of the status quo, or offer a better improvement plan.

Sorry but you don't get to set the terms of your own critique. If you refuse to listen then that impugns your character, but it doesn't make the critique invalid.

You method is flawed because some people only leave reviews when they good service or only when they get bad service. Everyone has a different reason why they do things, and what motivates them to do things. This will only work if you force people to leave reviews, and in that case your chances of getting a legitimate rating is much smaller.

You system would be better suited to de-rate the importance of people who leave a lot of reviews that are consistently skewed towards one end, but apply such a system to everyone is not going to make the results more fair.

> If you want your 10 stars to mean 10 stars, you need to rate a bell curve of other stuff too.

If you want your positive rating of something to have the most impact (for whatever reason), then you need to give negative ratings to other things (even if they deserve better)?

Maybe this could help with rating inflation by legitimate reviewers, but I don’t think it would help (and could possibly hurt) with the problem of fraudulent reviews. It would just mean that fake reviews would counterbalance their fake positive reviews with fake negative reviews of other products, making all reviews, in both directions, totally unreliable.

a five-star system, but with stars instead of labels.

"Absolutely awful service" / "dissatisfied" / "basically what I expected" / "positively surprised" / "incredibly excellent"

This way, most customers would click "basically what I expected", making 3 star the default.

Adjusting all reviews based on the general mean and standard deviation after the fact makes sense, though.

Stuff like this doesn’t naturally fall on a bell curve. Most hotels are great. Most restaurants are pretty good. There is no such thing as an objective 5.

in a normal distribution, you expect the mode to be at the mean to be at the median. If that's where most hotels are, and most restaurants, welcome to what 5 objectively means.

You expect that people will rate most things to be 5s based on averages, since if you aggregate your best experiences as 10s and your worst as 0s, then most everything is a 5, right?

But human brains don't work like that. We don't compare a dinner at a restaurant to literally the best dining experience we ever had or a trip to a theme park with the birth our child, so you can't realistically expect people to operate in that way when filling out a review form.

yes, and that's why figuring out the distribution of your opinions and normalizing them to convolve them with the rest of the raters will extract useful information from your ratings.

According to your logic we should be failing half of all school classes because half the kids shouldn't be above average.

Not many people are going to think that a system which weights everything to be distributed that way is useful as a metric for evaluating human experiences.

Reply on news.ycombinator.com