Ask HN: What is the best rating system? | Hacker News Reader

Ask HN: What is the best rating system?

14 pointsjpn9 comments

Bonjournal is a journaling app that helps you document and share your travels.

https://bonjourn.al

We are thinking of adding a rating system so that users can rate the places they visit. Think journaling meets Yelp / Foursquare.

Obviously no rating system is going to be perfect. Yelp uses the five star system. Foursquare uses a 10 point scale. We are leaning toward letter grades.

Why I like letter grades:

1) Precision: A, A-, A-/B+, B+ 2) Less ambiguous: I have a strong understanding of what a B vs a B+ means...not so much with 6.5 vs 7 or 3.5 stars vs 4. Perhaps that is just me. 3) Widely adopted: http://en.wikipedia.org/wiki/Grading_systems_by_country

However, we're not 100% sold, and wanted to ask the HN community.

* What rating system do you prefer and why? *

Some references:

http://techcrunch.com/2009/09/22/youtube-comes-to-a-5-star-realization-its-… http://www.quora.com/Is-there-a-better-alternative-to-the-5-star-rating-sys… https://www.quora.com/How-reliable-is-the-100-point-scoring-system-for-choo…

Shannon Appelcline has a great overview of rating, ranking, and other systems in the article series "Collective Choice": http://www.skotos.net/articles/TTnT_/TTnT_178.phtml . The key points are that most people want to be nice so you'll have far more activity in the top half of the ratings (3-5 for 5 stars, 6-10 for 10 stars); explicitly describing what each level means makes people more accurate; and instead of using a simple average you should weight it towards the site average to prevent items with few ratings from being misrepresented (e.g. an item with one 5-star rating is not the best ever).

Whatever scale you use, this article has good advice on how to compute each item's total score for sorting purposes:

http://www.evanmiller.org/how-not-to-sort-by-average-rating....

Thanks. This is a great find.

aside: I was going to refer the goodfilms blog post[1] but decided to check quora first (I've stopped using it because of their login/read only first answer) and there it was! (and it appears that quora has removed the gating)

Also, can you make your references clickable?

[1] http://goodfil.ms/blog/posts/2011/10/07/a-better-way-to-rate...

I think it would be better to ask the user to compare a place against a few others he has visited and use that to automatically compute a rating. But thats not as usable.

Another option would be to ask him to fill in a few questions.Then compute the rating using that.

Building Web Reputation Systems:

http://shop.oreilly.com/product/9780596159801.do

I like how Amazon does it, with stars and comments. That way you can look at the stars given and then add in the grain of salt by the quality of the associated comment.

Amazon also gives (or used to give) you a helpful tooltip of what they want each star to mean. I think they were: I love, I like it, It's okay, I don't like it, I hate it. I've started using it for 5-point scales and a with a slight adjustment for 10-point scales. If I would say one of those things about an item that is the rating I give.

Another good thing about Amazon's ratings is that you can see a breakdown of points. A 3-star item with 20 3-star reviews is likely to be quite different to an item with 10 1-star and 10 5-star reviews.

Although if I was a "professional" doing reviews, I would probably use a grade system of A-C and F.

I actually like 5 star rating system for simplicity with an added ability to rate at 1/2 star intervals if wanted to.

Reply on news.ycombinator.com