Comment by Jun8 | Hacker News Reader

Comment by Jun8 | Hacker News Reader

Parent Full thread

Maybe it's a Saturday night thing but the discussion for this part mostly has comments that rival the ignorance of the OP, e.g. "The sad fact is that most people are awful", "you could block that country from your app", all based on simplistic anecdotal analysis given in the post.

Before you arrive at such a sweeping conclusion, you should perform at least a little bit of statistical testing, e.g. see diziet's analysis on this page.

Two further comments:

1. The behavior mentioned in the post does seem to be a real thing, as evidenced by the many entries in the Turkish zeitgeist site eksisozluk: https://eksisozluk.com/yorumum-gozuksun-diye-1-yildiz--41109..., the first entry under this heading is dated 11/13/2013 so it seems to be a new meme. Also, the entries suggest that the behavior seems to be focused on Google Play rather than the Apple Store.

2. This shows a significant drawback of Google Play's selection of comments to display. Looking at the selected comments for the same game on the Apple Store (https://itunes.apple.com/us/app/wheres-my-water-2/id63885314...) I don't see this kind of 1 star ratings.

You wouldn't see these ratings on the Apple Store unless you are in Turkey (or changed your device's region) because Apple separates app store ratings. US devices only see US ratings. Turkey only sees Turkey ratings.

EDIT: Also, Apple doesn't display ratings like Google does, so there is clearly no reason for them to leave reviews like that.

That's clever. It accounts for cultural differences in rating systems.

A 3/5 isn't a good rating in any country I've lived. A 5 is good. There isn't really a rating for "exceptionally amazing". But I'm sure somewhere, a 3/5 means "pretty good" and 5/5 is reserved for exceptionally positive reviews.

May be my Norwegian upbringing or that I now live the UK, but when I like an app I often give it 3/5 stars, which to me is "not bad" (which means "good" in the UK). 5/5 would be amazing life changing rating and I don't get that excited by any app. I have give 4/5 to a few apps that is installed on all my devices.

So the problem with world wide rating is that are so different depending on your location and culture (and age).

Although I only install apps that have 4.5 star rating so I am being hypocritical...

Anecdotal tangent: A long time ago when I was in school in Norway, grades was marked on sort of 1-5 rating. (LG=Not very Good, NG=Quite Good, G=Good, MG=Very Good, S=Exceptionally Good. http://no.wikipedia.org/wiki/Karaktersystemer_i_Norge#Grunns...) As I was usually among the best I mostly got a grade in the range of MG ie a 4. The top rating was practically impossible and I only got that on a few occasional tests but never on a full year's grade. MG+ was the more likely top grade to aim for. (They have since changed and expanded the grade system)

When I then moved to the UK in my last years of schooling, grade A seemed to be handy out all over the place. I do remember having to have meetings with the head of the year to be put into the best Maths set even if my average from Norway was not of the top grade and trying to explain that top grade in the two countries was not equivalent.

I think the purpose of the rating system motivates how people rate. Since the App Store appears to give importance to mean average data, I am given motivation in that direction:

If I think the app is good, and other people should download it, my rating makes the most difference if I award a score of 5.

If I think the app is OK but not worth downloading, I can most effectively indicate that by scoring it a 1.

If I want people to read my review (as Turkish reviewers have observed), I should perhaps rate it a 1 as well.

So, if I want my reviews to have an observable effect, I should (perhaps) rate everything 1 or 5.

(Or, I could decide what rating I want the app to have, then rate it a 1 or 5 depending on which direction I need to move it)

The motivation initiated by Apple here is bad - by simplifying everything to a single figure average, we lose all the nuance that you are trying to give with your Norwegian-style ratings.

> (Or, I could decide what rating I want the app to have, then rate it a 1 or 5 depending on which direction I need to move it)

You just re-discovered strategic voting. And that's why the rating system should display the median and not the mean. For the median, you can skip step two: just vote what you want.

I also have a Norwegian upbringing. If I like an app (and consider it "good"), then I give a 5/5 rating. If I like it, but find a few flaws in it, then I give a 3/5 or 4/5. As for 2/5 and 1/5, that means I don't think it's worth it for most other users.

Interesting. Maybe app stores should show a helpful chart to explain the star rating, so that the results are more uniform.

In my case,

5 stars = Does exactly what I want; 4 stars = Does most of the things I want, minor features missing; 3 stars = Has a few useful features, the rest are irrelevant; 2 stars = Frequent crashes and major bugs; 1 star = Does not install or start at all no matter what I try (very rare).

I mostly give 4 stars, but there are a lot of 5 star worthy apps, as well (Pocket Informant and ES Explorer, for example).

I don't believe you are a hypocrit for realizing the bulk of voters believe 5 is "good," and adjusting accordingly.

I live in Germany, where the old people all like to critizise everything and the young people are as enthusiastic about apps as any American.

Bimodal rating distributions everywhere. Help!

It's not necessarily that people tend to have extreme feelings about an app - it's just that if they don't feel strongly about it (positively or otherwise), they don't bother to vote.

Similarly, as a Brit I'm not sure whether I've ever given an app, or film, or whatever a 'perfect' score. Really enjoyed it? A sold 4 from me.

Edit: Never mind, I made a mistake reading your comment.

I ran into this cultural differences while hiring and they are significant. Did you live in France? 20 is for the pope, 19 for the president and 18 for the headmaster, they say.

I think you have to say that grading in France is usually over a 20-point scale. And indeed, the highest grades are rarely attributed even for near-perfect work.

Irish here. We mark things based on a percentages, and it would be rare to give someone very high marks (like 95%+). Then again our pass rate is 40%, and 85%+ is an A, so it could be scaled down. I think the USA uses different scales.

What happens to the 60% who fail? Do they retake the class or drop out and look for a job?

The grandparent means 'passing grade', not 'pass rate'.

Yes. Perhaps a dialect difference. We'd say "pass rate" to mean "get less than this and you fail, more and you pass". We don't tend to use the word "grade" here for results in tests, to my ears it sounds very Amercian. ☺ We'd say "mark".

Thanks! I couldn't quite wrap my mind around a school with 60% per-period attrition.

At least in science it is pretty common (and many teacher grade over 25 so even near perfect can give you a 20).

Lower is better?

I don't think there's a country like that. Users are lazy and I think 5/5 is "it works well according to me" everywhere and next rating is 1/5 for "it doesn't work/it sucks" for most. Even I do this at times. Most of my ratings are 5 and 1.

Not just cultural differences.

I've always hated Yelp and found it useless until I went on holiday to US. In Australia, there's just so few places on it its kinda boring.

I'm still not a big fan of Yelp (you can buy a good rating), but it's the only popular review app that I know of.

How can you buy a good Yelp rating?

There are reports that Yelp ad sales people claim if you buy advertising with them they can make bad reviews go away [1]. Of course, it's possible the sales people are just claiming that to make the sale and don't actually deliver on it.

There are also people on ebay [2] offering five star reviews for around $15.

[1] http://www.eastbayexpress.com/oakland/yelp-and-the-business-... [2] http://www.ebay.com/sch/Web-Computer-Services-/47104/i.html?...

No it is not clever. Is is beyond idiotic since the system tries to determine the culture I'm interested in for me - without allowing me to easily change that.

You can easily change it - you just click the flag at the bottom of the iTunes store and switch to the country of your choice.

Actually, it's quite frustrating. It means that for many (geeky, 'international') apps that I'm interested in there's no rating available.

In Germany it's the opposite

A 1/5 would be the highest score, and 5/5 would be the worse score.

But only for school grades which go from 1 to 6 (with both 5 and 6 failed).

I've seen several customer satisfaction questionnaires like that

Living in Germany I'd rather say both systems are used. 1 best, 5 worst is supposed to reminiscent of school marks and 5 best, 1 worst is reminiscent of Amazon ratings. That's why normally in questionnaires you write next to it what is supposed to be best and what worst. Intuitively I'd indeed choose the 1 best, 5 worst system for surveys in Germany, but it's not consistently used.

for the record in the us:

5* good 4* ok but has some issues 1* sux 2 and 3: irrelevant

You're right. Here's the link to the game on Turkish Store: https://itunes.apple.com/tr/app/wheres-my-water-2/id63885314...

Hi, I am aware that the post on my blog is pretty shallow but my intention was not to dive deeper into the problem anyway.

As you confirmed, the phenomenon is real. I am an IOS user and I was shocked when a friend told me about the comments people are making at Google Play, so I quickly put together a blog and wrote a post that dramatically exposes the problem. The idea is to catch attention of the people so that those who have the necessary tools and time can investigate it thoroughly.

The problem is real and hopefully somebody in the position to fix it would see the post.

Sample size for Where's My Water is pretty small, though. 6 reviews in the last 90 days in the Turkish Store ( https://sensortower.com/ios/us/disney/app/wheres-my-water-2/... )

Compared to the US, which received a lot of negative reviews (https://sensortower.com/ios/us/disney/app/wheres-my-water-2/...) and has a larger sample size. It could be a much more recent behavior.

Reply on news.ycombinator.com