Comment by supertrope | Hacker News Reader

Comment by supertrope | Hacker News Reader

Parent Full thread

There was always a problem with misaligned incentives. If you're not paying for the review, they have no loyalty to the reader.

e-commerce websites pay lip service to deleting fake reviews but higher product review scores result in more sales. Even if this doesn't result in intentionally ignoring fake reviews in pursuit of short term sales growth, note how Amazon.com like all other five star rating systems suffers from massive score inflation. Most have a 4.5/5.0 rating. 4.0 indicate some potential problems (or less sophisticated customers), and 3.5 means it's sub-par.

A lot of online stores simply have no customer review section because they have rationally determined that for them decreased conversions due to bad reviews and moderation costs exceed increase in conversions due to good reviews.

> five star rating systems suffers from massive score inflation

Indeed.

I liked how Goodreads had their original rating scale done: 3 was good, 4 was very good, and 5 was truly exceptional. You weren't supposed to give 5 to more than maybe 2% of the books you'd read.

Then the got bought out (by Amazon) and the scale got soon enough devalued.

I still think that one potential solution to the scale inflation would be to consider the grade distribution a person uses. If all they give out is ones or fives, their reviews should have near zero weight. If they give out a more balanced (on a ~Gaussian curve) reviews, then their rare extremes should be weighted much higher.

Your idea is along the lines of 'rater reliability' in the field of educational measurement/psychometrics.

Honestly just make it thumbs up or thumps down. Five star ratings are quite pointless.

I like Rotten Tomatoes's system. Individual critics give a thumbs up or down. A movie is "certified fresh" if at least 80% think it was worth watching. It doesn't mean the average rating was 80% or that that number means it was amazing or excellent, it just means the vast majority thought it was worth your time and money.

I recently used a site called “Tabelog” while traveling in Japan - the median review is roughly a 3.2, a pleasant surprise that left me with a strong insight that a restuarant with a 4 or above was quite a quality establishment.

Reply on news.ycombinator.com