Awful app review trend among Turkish users | Hacker News Reader

Awful app review trend among Turkish users

blog.appwared.com

352 pointsmrtksn198 comments

Open articleView on HN

On iOS, we have indexed 666,355 reviews in Turkey. Out of those, 92,853 are 1 star reviews. 13.93%

On iOS, we have 32,296,341 reviews, with 3,448,351 being 1 star reviews. 10.67% one star reviews.

Let's see if we limit it to reviews in the last 2 months:

16.47% negative in Turkey (65,154 total reviews) 12.05% negative in the US (1,960,283 total reviews)

(source - internal https://sensortower.com analytics)

It's a bit harder to separate reviews between countries on Android, but we have 7,862,505 recent reviews indexed in the last 2 months on Android, and 780,997 of them are negative (9.93%). I will try to run a query with language detection, though that might take a bit of time~

Thank you, this is cool. It's very annoying to read comments like "I gave 1 star but it's a good game" however I wonder how much is the actual effect on the ratings.

I'm seeing a similar number for iOS:

    Rating         % All         % Turkish
    1              09.6%         15.2%
    2              03.5%         04.4%
    3              06.5%         07.0%
    4              16.6%         14.5%
    5              63.6%         58.7%

(Source: data from MixRank's iOS intelligence)

There's definitely a bump in 1-star reviews in Turkey.

Could be accounted for by apps with poor localization.

So these apps have approximately equally good localization for every other language, except Turkish?

I get that you were being flippant, but apparently Turkish is unusually hard to localize for: http://www.codinghorror.com/blog/2008/03/whats-wrong-with-tu...

This post is a little like programming pop science: an entertaining read, but greatly overstates some small result. The only Turkish-specific problem he mentions is case folding of ı/i. This is a problem unique to Turkish, there are other problems in case folding, like eszett (ß/ss) in German but these are much less likely to trip you up.

The US date format (m/d/y) is used virtually only in US, most other places use d/m/y or y/m/d. -- http://en.wikipedia.org/wiki/Date_format_by_country#Map

Re dots for thousands separators -- the world is split about roughly evenly on this, with all of europe (except for Great Britain and Ireland) using commas for decimals dots for thousands separators. -- http://en.wikipedia.org/wiki/Decimal_mark#Hindu.E2.80.93Arab...

"This post is a little like programming pop science" That's Jeff Atwood for you.

That bump may be bigger if you separate by date. If it's a new trend then the newer Turkish reviews will be the outliers.

> There's definitely a bump in 1-star reviews in Turkey.

Wait wait wait, the mere fact that Turkish has more 1 start review than the world average doesn't tell you anything about other countries individually.

It could be that most countries are above the world average except a couple of countries with almost no 0-star reviews.

If you want to make this point, at least give us the standard deviation.

That's why we don't have nice things in Turkey.

> 666,355

For a second I misread that as 65,536 (64K) reviews.

Maybe it's a Saturday night thing but the discussion for this part mostly has comments that rival the ignorance of the OP, e.g. "The sad fact is that most people are awful", "you could block that country from your app", all based on simplistic anecdotal analysis given in the post.

Before you arrive at such a sweeping conclusion, you should perform at least a little bit of statistical testing, e.g. see diziet's analysis on this page.

Two further comments:

1. The behavior mentioned in the post does seem to be a real thing, as evidenced by the many entries in the Turkish zeitgeist site eksisozluk: https://eksisozluk.com/yorumum-gozuksun-diye-1-yildiz--41109..., the first entry under this heading is dated 11/13/2013 so it seems to be a new meme. Also, the entries suggest that the behavior seems to be focused on Google Play rather than the Apple Store.

2. This shows a significant drawback of Google Play's selection of comments to display. Looking at the selected comments for the same game on the Apple Store (https://itunes.apple.com/us/app/wheres-my-water-2/id63885314...) I don't see this kind of 1 star ratings.

You wouldn't see these ratings on the Apple Store unless you are in Turkey (or changed your device's region) because Apple separates app store ratings. US devices only see US ratings. Turkey only sees Turkey ratings.

EDIT: Also, Apple doesn't display ratings like Google does, so there is clearly no reason for them to leave reviews like that.

That's clever. It accounts for cultural differences in rating systems.

A 3/5 isn't a good rating in any country I've lived. A 5 is good. There isn't really a rating for "exceptionally amazing". But I'm sure somewhere, a 3/5 means "pretty good" and 5/5 is reserved for exceptionally positive reviews.

Not just cultural differences.

I've always hated Yelp and found it useless until I went on holiday to US. In Australia, there's just so few places on it its kinda boring.

I'm still not a big fan of Yelp (you can buy a good rating), but it's the only popular review app that I know of.

How can you buy a good Yelp rating?

There are reports that Yelp ad sales people claim if you buy advertising with them they can make bad reviews go away [1]. Of course, it's possible the sales people are just claiming that to make the sale and don't actually deliver on it.

There are also people on ebay [2] offering five star reviews for around $15.

[1] http://www.eastbayexpress.com/oakland/yelp-and-the-business-... [2] http://www.ebay.com/sch/Web-Computer-Services-/47104/i.html?...

I ran into this cultural differences while hiring and they are significant. Did you live in France? 20 is for the pope, 19 for the president and 18 for the headmaster, they say.

I think you have to say that grading in France is usually over a 20-point scale. And indeed, the highest grades are rarely attributed even for near-perfect work.

Irish here. We mark things based on a percentages, and it would be rare to give someone very high marks (like 95%+). Then again our pass rate is 40%, and 85%+ is an A, so it could be scaled down. I think the USA uses different scales.

What happens to the 60% who fail? Do they retake the class or drop out and look for a job?

The grandparent means 'passing grade', not 'pass rate'.

Thanks! I couldn't quite wrap my mind around a school with 60% per-period attrition.

Yes. Perhaps a dialect difference. We'd say "pass rate" to mean "get less than this and you fail, more and you pass". We don't tend to use the word "grade" here for results in tests, to my ears it sounds very Amercian. ☺ We'd say "mark".

At least in science it is pretty common (and many teacher grade over 25 so even near perfect can give you a 20).

Lower is better?

Actually, it's quite frustrating. It means that for many (geeky, 'international') apps that I'm interested in there's no rating available.

for the record in the us:

5* good 4* ok but has some issues 1* sux 2 and 3: irrelevant

No it is not clever. Is is beyond idiotic since the system tries to determine the culture I'm interested in for me - without allowing me to easily change that.

You can easily change it - you just click the flag at the bottom of the iTunes store and switch to the country of your choice.

I don't think there's a country like that. Users are lazy and I think 5/5 is "it works well according to me" everywhere and next rating is 1/5 for "it doesn't work/it sucks" for most. Even I do this at times. Most of my ratings are 5 and 1.

May be my Norwegian upbringing or that I now live the UK, but when I like an app I often give it 3/5 stars, which to me is "not bad" (which means "good" in the UK). 5/5 would be amazing life changing rating and I don't get that excited by any app. I have give 4/5 to a few apps that is installed on all my devices.

So the problem with world wide rating is that are so different depending on your location and culture (and age).

Although I only install apps that have 4.5 star rating so I am being hypocritical...

Anecdotal tangent: A long time ago when I was in school in Norway, grades was marked on sort of 1-5 rating. (LG=Not very Good, NG=Quite Good, G=Good, MG=Very Good, S=Exceptionally Good. http://no.wikipedia.org/wiki/Karaktersystemer_i_Norge#Grunns...) As I was usually among the best I mostly got a grade in the range of MG ie a 4. The top rating was practically impossible and I only got that on a few occasional tests but never on a full year's grade. MG+ was the more likely top grade to aim for. (They have since changed and expanded the grade system)

When I then moved to the UK in my last years of schooling, grade A seemed to be handy out all over the place. I do remember having to have meetings with the head of the year to be put into the best Maths set even if my average from Norway was not of the top grade and trying to explain that top grade in the two countries was not equivalent.

Similarly, as a Brit I'm not sure whether I've ever given an app, or film, or whatever a 'perfect' score. Really enjoyed it? A sold 4 from me.

Edit: Never mind, I made a mistake reading your comment.

I don't believe you are a hypocrit for realizing the bulk of voters believe 5 is "good," and adjusting accordingly.

I live in Germany, where the old people all like to critizise everything and the young people are as enthusiastic about apps as any American.

Bimodal rating distributions everywhere. Help!

It's not necessarily that people tend to have extreme feelings about an app - it's just that if they don't feel strongly about it (positively or otherwise), they don't bother to vote.

I think the purpose of the rating system motivates how people rate. Since the App Store appears to give importance to mean average data, I am given motivation in that direction:

If I think the app is good, and other people should download it, my rating makes the most difference if I award a score of 5.

If I think the app is OK but not worth downloading, I can most effectively indicate that by scoring it a 1.

If I want people to read my review (as Turkish reviewers have observed), I should perhaps rate it a 1 as well.

So, if I want my reviews to have an observable effect, I should (perhaps) rate everything 1 or 5.

(Or, I could decide what rating I want the app to have, then rate it a 1 or 5 depending on which direction I need to move it)

The motivation initiated by Apple here is bad - by simplifying everything to a single figure average, we lose all the nuance that you are trying to give with your Norwegian-style ratings.

> (Or, I could decide what rating I want the app to have, then rate it a 1 or 5 depending on which direction I need to move it)

You just re-discovered strategic voting. And that's why the rating system should display the median and not the mean. For the median, you can skip step two: just vote what you want.

Interesting. Maybe app stores should show a helpful chart to explain the star rating, so that the results are more uniform.

In my case,

5 stars = Does exactly what I want; 4 stars = Does most of the things I want, minor features missing; 3 stars = Has a few useful features, the rest are irrelevant; 2 stars = Frequent crashes and major bugs; 1 star = Does not install or start at all no matter what I try (very rare).

I mostly give 4 stars, but there are a lot of 5 star worthy apps, as well (Pocket Informant and ES Explorer, for example).

I also have a Norwegian upbringing. If I like an app (and consider it "good"), then I give a 5/5 rating. If I like it, but find a few flaws in it, then I give a 3/5 or 4/5. As for 2/5 and 1/5, that means I don't think it's worth it for most other users.

In Germany it's the opposite

A 1/5 would be the highest score, and 5/5 would be the worse score.

But only for school grades which go from 1 to 6 (with both 5 and 6 failed).

I've seen several customer satisfaction questionnaires like that

Living in Germany I'd rather say both systems are used. 1 best, 5 worst is supposed to reminiscent of school marks and 5 best, 1 worst is reminiscent of Amazon ratings. That's why normally in questionnaires you write next to it what is supposed to be best and what worst. Intuitively I'd indeed choose the 1 best, 5 worst system for surveys in Germany, but it's not consistently used.

You're right. Here's the link to the game on Turkish Store: https://itunes.apple.com/tr/app/wheres-my-water-2/id63885314...

Hi, I am aware that the post on my blog is pretty shallow but my intention was not to dive deeper into the problem anyway.

As you confirmed, the phenomenon is real. I am an IOS user and I was shocked when a friend told me about the comments people are making at Google Play, so I quickly put together a blog and wrote a post that dramatically exposes the problem. The idea is to catch attention of the people so that those who have the necessary tools and time can investigate it thoroughly.

The problem is real and hopefully somebody in the position to fix it would see the post.

Sample size for Where's My Water is pretty small, though. 6 reviews in the last 90 days in the Turkish Store ( https://sensortower.com/ios/us/disney/app/wheres-my-water-2/... )

Compared to the US, which received a lot of negative reviews (https://sensortower.com/ios/us/disney/app/wheres-my-water-2/...) and has a larger sample size. It could be a much more recent behavior.

If you are going to take user reviews seriously, you have to account for the possibility that a significant portion of the general population may be comprised of idiots.

People misusing the star system is a problem, but the real problem is that we are forced to care when people misuse it.

(To correct this specific problem, I would recommend low-tolerance hellbanning. If a positive review is accompanied by a one-star rating, that user's ratings and comments will never be seen by another user again.)

Are developers on the play store allowed to delete reviews? That sounds like something that would be really easily abused (i.e. delete all bad reviews of your bad product).

There is no option on the developer console for this afaik. If there is it is not prominent or easily discovered.

However, I saw a small burst of bad reviews on an app I used to work on a few months back - all of those reviews including the developer replies are now 'missing'.

Its possible they merely flagged the reviews and Google got rid of them because they were rude. Possibly the OPs removed them because the developer got in contact to ease their pain somehow...

I am Turkish, I live in the US and run a startup. I conducted a small experiment (n=3) among my cousins living in Turkey aged 8-14: They don't understand that 3rd parties make these apps not tied to the device; and they definitely don't understand that the rating (in general) would impact someone making this app financially. Huh.

Why do you think this is more prevalent in Turkey than in other countries? Is there something unique about Turkish culture that would cause this misunderstanding or misprioritization?

I suspect the misconceptions are wider spread than Turkey, though the meme of giving one star reviews to be "at the top" has yet to spread.

Hopefully this article, and others like it, do not give the meme a bump up across the language barrier.

>> "Why do you think this is more prevalent in Turkey than in other countries?"

If you're referencing the fact that they don't understand the apps are made by third parties it isn't limited to turkey. I'm an iOS developer and the number of people that think I 'work for Apple' is incredible.

One of my older relatives thinks I work "for the internet".

A master of the web. A webmaster.

Just to be clear, are you saying that they think Google makes all the apps in Google Play?

Sadly, I've heard that many non-tech-savvy people in the US believe Apple makes all the apps in the AppStore. "Oh you make iPhone apps? How do you like working at Apple?!!"

It's so sad when people are that stupid :(.

As a child my father had a laptop with OS/2 on it, and on the laptop was an OS/2 sticker. So for me my father's laptop and OS/2 were interchangeable, even synonym. Then one time he showed me a bunch of floppy disks and tried to explain, that those floppies are OS/2 ;-)

I don't think this phenomenon has much to do with user misconceptions, it's more of a meme that has spread to many Turkish users, e.g. do a search on "gozuksun diye 1 yildiz verdim" (gave 1 star so my comment is seen) or the eksisozluk link in my comment below.

So you say that you asked a bunch of questions to a couple kids and concluded that the whole population of Turkey are idiots? Let me tell you, your cousin's reactions are quite expected, but I doubt yours are.

http://en.wikipedia.org/wiki/Scientific_method

He simply provided a data point, mentioned the sample size, and indicated their responses surprised him. There is no generalization, nothing indicating the conclusion of the sorts you mention. I'm baffled by your interpretation of his comment. "concluded that the whole population of Turkey are idiots?" What makes you think that?

" your cousin's reactions are quite expected,"

Why?

Before you get all uppity about the scientific method, I suggest you read http://en.wikipedia.org/wiki/Jumping_to_conclusions

Even without the context of my comment, this still makes a great read. Thank you. BTW, I have to admit that I have misinterpreted the comment, my apologies.

Turkish users want their reviews to be seen. This sounds narcissistic to us, but for a lot of people being asked for their opinion of a product is a big deal. Rather than condemning the users, we should look for ways to improve the UX of the Google Play Store.

There could be cultural factors at play here as well. When I was growing up in Turkey, every so often there would be a fad that picked obscure artefacts of American culture and turned them into articles of social prestige. Cheesy American soap operas, songs and actors would suddenly and inexplicably become national sensations that everyone had to know. I assume something similar is going on with the Android Play Store reviews. Having your review on the frontpage of an app had become a point of pride, so much so that users are subverting the system to get there.

What's interesting is that Google Play will already automatically surface the reviews from anyone in your extended circles.

It's kind of weird that they want their comment to be at the top, but the only thing they say in their comment is that they want it to be at the top. You'd think if they were looking for exposure, they'd use the exposure to do something other than talking about how they wanted exposure.

First.

Priceless.

I think the First Post meme has a bit of orthogonal justification to it, though: that if you spend the time to write something other than "first post," someone else will have beaten you to posting by writing less.

I didn't even know that was ALLOWED on Hacker News! All this time I've been going back to SlashDot when I needed to post that, and I could have been claiming first posts here.

I suspect it's not normally allowed or encouraged, but I think it is uniquely relevant and appropriate in this case.

I wasn't seriously suggesting otherwise ... It was supposed to be funny (and to subtly make the point that we really don't want that here).

You got it. My mother made $27,177 last month just sitting at home answering the phone.

Haha, funny thing: I've seen comments greetings their loved ones. Like it's on TV and their relatives are watching it :)

you've clearly never used slashdot (:

Are you joking or are you really this naive?

This seems like a great channel for spammers, but I suppose Apple/Google would permanently ban the user's account quickly.

In a mass used platform a single user should never be able to receive 100% visibility by a simple action.

This makes me wonder... are there locales where 1 star might be taken to mean a better review than a 5 star review?

I realize that's not what's being claimed here, though.

There's no intrinsic meaning to how stars correlate to quality is my thinking here, but perhaps only due to familiarity do we think that more stars is better and not worse.

It seems at least in this case they know what they are doing because they say thing like this "Great game, but give it 1 start so people see my comment". I think they are fully aware and are even "apologizing" for it.

Here's a comment which suggests that at least some of them are thinking 1 star is the best:

Kubilay Tekin: I hadn't gave 1 star just because I want my comment gets shown on top, I actually think that the game is aweful aqws.w.wew..ws

(I've seen this phenomenon before - there are quite a few who associate 1 with first place, hence surveys and such often have the [1 = worst, 10 = best] next to each "rate xxx on a scale of 1 to 10")

I don't think that's what that comment suggests. The commenter is acknowledging that some people give one star just to have their comment seen, even though they think it's a good app, but that he personally is giving one star because he thought it was indeed awful.

I saw it as that he is giving 1 star because he wants the comment to be seen, but actually thinks it was awful (implying that he wouldn't give 1 if it wasn't awful). Either way the grammar certainly taxed my parser...

You misread that. It says "I hadn't gave 1 star just because I want my comment gets shown on top"

He is saying he hasnt only given 1 start to be at the top, he actually does think the app is terrible

What? No, it doesn't suggest that at all. The reviewer actually writes that he thinks it's awful and that he would have given it 1 even though he didn't want his comment to be on top.

Indeed there are many rating systems where '1' is 'best', 'first-in-rank', 'most-preferred'.

I've never seen an online 'stars' system that works that way... but it's possible in some other cultural context a single star is somehow honored more than many, and thus without explanation the star-system could be misinterpreted. (Of course the comments here strongly suggest that's not a factor in this case.)

I am from Turkey and no, 1 star is still 1 star here. It is not a cultural mistake, it is just the people in example are being narcissistic and trying to show off

http://nlp.stanford.edu/sentiment/

> Good, a very good game! I gave 1 star so that my comment gets seen.

++ very positive

> The game is an example of how a good game should be! Don't bother that I gave 1 star, it's because I want my comment gets seen.

++ very positive

> Super! I gave 1 star so that my comment is shown first but I actually liked the game.

0 neutral

> I hadn't gave 1 star just because I want my comment gets shown on top, I actually think that the game is aweful aqws.w.wew..ws

- negative

Maybe star ratings should be removed in favor of text reviews with "ratings" based exclusively on sentiment analysis. Has machine learning reached a state where that's practical? If not, will anyone hazard a guess on when it will?

Just offer both, and weigh them behind the scenes however you feel like.

WTF? Why is this bug report getting spammed with talk about Turkey? The bug is that the system displays one star reviews first, creating broken incentives. Assuming that the code works (i.e., doesn't work) the same way in every country, the country has nothing to do with it. The system needs to be fixed to use a more neutral display order.

Is there anyone else who thinks the "5 star" rating is somewhat flawed, or is it just me?

I think, the problem is that the scale is all on the positive side, but people are trying to use it to give negative rating. So naturally they readjust the scale, so that it has both, positive and negative sides, by "shifting" the zero of the scale. Now, where exactly does it go? Will 3 become 0, or is it 2 for some people, or maybe 4?

I know it's way too late in the game, but why can't the rating system be just a simple "good", "bad", "it's ok", or maybe even: :-(, :-|, :-)

I am not enamoured with the "5 star" system, but I have to say that I like it more than the standard 5-choice system that explicitly labels the choices as positive or negative: Strongly Disagree, Disagree, No Opinion, Agree, Strongly Agree

I despise that sort of system. None of the choices ever seem to be appropriate for what I think or feel.

Star systems avoid inspiring this frustration in me by giving me the freedom to imagine labels that I think fit how I feel or think.

>Star systems avoid inspiring this frustration in me by giving me the freedom to imagine labels that I think fit how I feel or think.

And in statistical circles, this is called survey bias (rating bias).

If you think a 4 is "awesome", and another person thinks that it means "pretty good", and this effect is widespread, it will result in a biased survey that is difficult to interpret.

Croatian school system uses grades 1 to 5. 1 is fail, 2 is pass, 3 is good, 4 is very good and 5 is excellent. Teachers sometime also add a minus or plus in front of the grade, e.g. -5 is almost excellent and 4+ is a bit better than very good.

When Netflix had their contest to improve recommendations, one of the common first-pass techniques on the forums was to zero-adjust the ratings for each user. It's hard to imagine that a big-data company of Google's size couldn't do this already, although who knows if they are.

Whatever netflix did it doesn't seem to have worked.

I noticed a while ago that literally 100% of netflix content (based on around ~50 things I checked on) has between 3 and 4 stars.

This makes ratings completely useless.

Interesting.

Is it for mixed reviews as in total reviews worldwide and Turkey or just for Turkey. It seems it is just for Turkey.

Maybe they are just more honest and say why they are giving 1 star reviews. Or maybe they are dishonest and really think the game is terrible but in the comment chose to say something else to seem nice? Is there somehow a disproportionate presence of Turkish Android app developers and since they are competing with you they are just messing with author's head.

Looking at the game review in the app store (presumably the US version) most 1 or 2 star reviews seem valid (and don't see a particular trend with names, to mean they are certain ethnicity).

A guess it would be good to browser other apps' reviews from Turkey. So I took a look at Cut The Rope. Very popular game, indeed.

Looking at US reviews, looks good.

Turkish:

https://play.google.com/store/apps/details?id=com.zeptolab.c...

Hmm, well I can't read Turkish. So I opened Google Translate and started translating a few top 1 star review. And yap, same pattern.

===

Mükemmel Çok güzel harika bir oyun gözüksün diye 1

---

Excellent Very nice people to see a great game as one

===

Gerçekten cok guzel bir oyun hem zeka açıcı Gercekten cok güzel

---

Really a very nice game're really nice and intelligence opener

===

Someone might want to help, but even with Google Translate it looks like they are giving it good reviews as text but 1 star as a score.

The crude and seemingly insensitive way is to just prevent Turkish reviewers from commenting. Or even better hellban them and never account for their score. The would correct this pretty quickly I would imagine.

Now I would really hope someone from Turkey to explain if there is a cultural or social reason for this. It just seems to strange and odd.

Hi, I can say Google translate did good enough job.

The speculation here in Turkey is that these are just kids. I see lots of complains from Turkish users on lot's of websites. Many Turks are switching to English just to get rid of this kind of comments.

> The crude and seemingly insensitive way is to just prevent Turkish reviewers from commenting. Or even better hellban them and never account for their score.

Maybe don't provide a guaranteed way to appear in the displayed comments? I'd expect this to spread outside of Turkey pretty quickly.

This could be better phrased. How about "don't require people to leave one-star reviews if they want to appear in the comments".

This is pretty sad for other Turkish people, who seeks proper reviews and know proper rating mentality. It is like hell for me. I translated an app to Turkish and wanted to check if there is any translation related comments. It blew my mind. There was 1 review in terms of new translation pack. The rest was 1-star reviews. I started to downvote these reviews (thumb down), but I guess no one cares enough to vote up or down useful reviews. There has to be another way. I guess voting anonymously will be the solution. Thanks to Google's Google+ integration, some stupid attention seekers want to show their full name on apps' first pages.

I don't even understand why a 1 star review would automatically appear, and a 5 star review wouldn't. What possible justification is there for that?

Anyway, Google has a few ways they could fix this:

* Allow users to rate reviews. Amazon does this. Why not Google? Or maybe rate reviewers, and don't show any reviews from notorious idiots. * Interpret the text of the review and check if it matches the score. Show those reviews that sound like they have something interesting to say. If any company can implement this, it's Google. * Separate reviews by country.

is the Turkish language read from right to left? maybe they are counting stars from the wrong side.

No, it's read left to right.

In 1928 Mustafa Kemal Atatürk started a plan[1] to ditch the old form of written Turkish (Ottoman Turkish which was RTL[2]) and implement one based on the latin alphabet in order to fight a huge illiteracy problem at the time.

[1]http://en.wikipedia.org/wiki/Mustafa_Kemal_Atat%C3%BCrk#Mode...

[2]http://en.wikipedia.org/wiki/Ottoman_Turkish_alphabet

Let me assure you, they know that 1 star is bad, 5 star is good. Even courses are rated between 1(bad)-5(good) in school reports. They also say that "I just rated 1 star to be seen, BUT it is a wonderful app", etc. I saw some comments like this : "I say hi to my cousin, this game is so fun. But I am giving 1 star to be on top." They are trolling and do not care or do not know about total rating system.

How did you conclude that "It seems it's just for Turkey". As far as I can see the OP has not done much analysis either and based his comments on a few comments. The only significant analysis I've seen is done by diziet below.

What I meant was that clicking that review link shows a localized page with Turkish reviews and his US (default?) review links doesn't show that anomaly.

But, it seems the total review average and review count is an aggregate of all localized review sites. So a pathological site will affect the average rating, without non-native speakers getting to read those ridiculous reviews and decided how unhelpful they are.

Sorry it was just my talking out loud figuring out how reviews on Play store are supposed to work.

As a Turkish Android user (not a developer, I just helped some developers by means of graphic design and Turkish language packs) let me speak my mind.

This thing is pretty new. I can't make a proper judgement of apps because of this new meme. I want to see international or English comments, but I can't without changing my language preferences. I guess it goes like this:

These people are mostly 10-15 years old, English-illiterate teens. Since Google uses real names on comments(Google+), they want to be seen with their names. I guess they show this to their friends, family, etc.. They feel like they are contributing maybe(?!) They seek for attention (?!) Or maybe it is just for trolling...

When you use Google Play on your PC, you can click on the names of the commentators. Maybe they think that it is a some kind of communication way :S Something like "add/PM me if you like". Weird.

When I check local reddit-like website of Turkey (eksisözlük), there are many people who are annoyed by this new meme. (https://eksisozluk.com/yorumum-gozuksun-diye-1-yildiz--41109...)

To sum up, I think this is really annoying, when I check the Google+ profiles of these 1-star-commentators, I see some non-english-speaking people or attention-seeking teens.

But what about Google Play's algorithm? Does it make sense? One-line-long comments with 1 star are being seen on front page of the apps... What about 2/3/4/5 starred useful comments? Why does Google Play show only 1-starred ones? It does not make sense at all. And these teens are just exploiting this. Why does Google Play's system accept 1-star comments as more helpful than others?

Likely, it's trying to show a helpful good and a helpful bad review next to each other. Amazon, etc all do this as well. I guess they haven't had as many issues with people exploiting it because you can rate reviews as helpful or not. Google should introduce a helpful or not helpful up/down rating for reviews which should solve this problem. Or a "This user is an idiot" button that, if enough people click, the review is thrown out. And, if enough people click it on multiple apps for the same user's reviews, they are banned from reviewing.

Actually there is such a helpfulness rating; thumbs up/thumbs down option for every review. I did not check on phone, but Google Play for desktop has that kind of system. I guess the problem is... nobody cares to vote.

I think the issue is most people don't use Google Play on the desktop to look at reviews and such. I have, I think, once. I generally only use Google Play on my Android devices.

This is really odd, particularly considering that Turkey is by far one of the friendliest places I've been in my many travels (well, at least many travels when I was younger).

I question whether they're aware of the grief that their reviews are causing. In any case, it's still pretty inconsiderate to tell someone (numerically) that you think their work sucks, even if you don't really mean any harm.

All in all, probably an extension of the whole "cursing/attacking/belittling someone on the Internet doesn't count" phenomenon.

They don't think that they are hurting developers, firms, etc. I saw a comment like this: "Hey dude, don't get me wrong I loved your app, it is really fun thank you; but I am giving 1-star just to be seen".

I guess they don't comprehend stars' purpose. When they are searching apps, they just scroll down to the comments. Stars are not a way to judge the apps. Also, with the Google+ integration, they enjoy their full name is being seen on an app's page. Weird.

Turks are friendly in person, and warm and welcoming. But in general, they don't seem to have as much of a general concept of civic duty. Eg they just litter, because there's no visible victim, and they don't see the guy who has to pick up the trash.

(I lived there for a few months.)

I hate the iOS review system. People rarely leave a review if the like the app, but as soon as they have something to complain about they will leave a bad review. Also most people if they have an issue will automatically leave a bad review instead of contacting the support email to get it resolved.

What's even more annoying is people that write things like 'Best app ever' and then give it a 3 or 4 star rating instead of 5.

Oh man this post is spot on LOL.

Man, with PC games you have forums so that's great for communicating, but with mobile it seems like there's generally a disconnect between developer and player. The system, while great to have, doesn't really encourage it.

Fortunately, developers have figured out an even more hateful response:

http://effyr.tumblr.com/

> Also most people if they have an issue will automatically leave a bad review instead of contacting the support email to get it resolved.

because most people are used to working with big products. Even me, a developer, this is the first time I considered that you could in general email support and not get some patronizing useless robot. It makes sense now that I think about it, since so many of the apps are made by small sized development firms or even just lone wolfs, there's a good chance I could get a human, maybe even another dev!

In general, that would never happen with a large and well known product. Faced with that, people take the only outlet they feel with maybe change something.

The best app ever part. Well...

People always exaggerate, ALWAYS.

As a developer you can combat both of these dynamics.

Make sure you provide a clear way for them to contact you if they are running into trouble.

Reach out to your engaged users and pro-actively ask them to review the app after you're confident they have had a good experience.

As a consumer, I find (honest) bad reviews far more useful than (even) honest good reviews.

I barely ever see reviews and have to ask to see reviews for other version. I'm in New Zealand. Its a disincentive to updating an app as far as I can tell. If something works and isn't broken, minor updates will lose the developer prominent reviews. Or am I missing something?

I think you'll see the same pattern all over the Internet, whether it's app stores or forums. People have to go out of their way to post something, and human nature is such that they're much more likely to expend that effort to post a negative opinion on something than a positive opinion.

I wonder if at least some of the motivation for "Please rate us!" popups in mobile apps isn't just to get more ratings, but also to get a more representative (i.e., more positive) sampling of user opinions as well.

We had the same problem, but with Russia. Ours being a paid app and our reviews being in the low 1000s, every 1-star rating was hurting us badly, so we removed Russia completely from our distribution. We added it back a month later and things went back to normal, but that's definitely something that we shouldn't need to have done.

Was it the same "awesome app but 1 star so I'm on the front page?". Or just widespread 1 star reviews solely from Russia?

Maybe Google should do sentiment analysis on review text and compare with star rating then rank the review to be less visible if the two contradicts too much?

If you aren't generating any revenue from Turkey you could block that country from your app.

As a Turkish, I think i know little better what is going on here;

It is the high school kids that they think it is cool to mess up with big and serious systems in their stupid way. It is fun for them to mess up rules which I think is the same for most of the teenagers around the world. They try to break things, don't listen teachers, don't go to school.

They think that they should be doing what they want instead of what others want them to do.

The outcome for the turkish teenagers irritating behaviour showed in Google's Play Store. That is all what i can see.

Aren't we all tired of these kids actions around us sometimes? :)

People's review habits in general are bizarre. I have read reviews like "I haven't gotten a chance to use the product yet" with a 3 star rating. One wonders what exactly they are reviewing. Or recipe reviews are the weirdest, every one of them say "I change blah blah blah" to the point where its a completely different recipe then they review their recipe, like that's somehow appropriate. I can understand making one tweek to suit your taste but don't change the whole recipe and think thats what you are reviewing.

When I FIRST released a paid game on the Android Market, after my first dozen 5-star reviews I was devastated by a 1-star review, the content of which was "I can't review this game in only 15 minutes."

And that user then promptly refunded the game within the (then-new) 15 minute window.

And (surprise!) someone ended up uploading a "cracked" [1] version of my game to warez sites shortly thereafter. Talk about adding insult to injury.

[1] I used a sneaky way to detect "cracks" in my game that involved, in part, letting the hacker think they'd cracked it: The game would query Google to see if they'd paid for it, and if not, it would throw a "Sorry, you don't own this game" dialog up. There were scripts around that would remove the standard Google DRM, though; instead of fighting that, I LET them "hack" it, and afterwards it wouldn't bring up that dialog any more. But then the behavior reverted to being identical to the free demo (first 30 levels free, then you had to buy the real game). It was "hacked" almost a dozen times, and never once did anyone figure out how to get past the second layer of protection.

How did you detect that the DRM check was removed?

Short answer: I looked at the SHA signature of the .class file (where all the compiled classes' bytecode is stored) and checked it against a known signature stored in an encrypted data file.

And the best part was the encryption on ALL the games' data files was based on a key derived from the binary executable (the .so file), so if anyone were to hack that executable, the game wouldn't run at all. I should have thrown up a screen that let people know that it was a cracked version, and where to get the real thing. And I DEFINITELY should have changed the Flurry code; my "paid game" analytics were completely shot after the cracks were released. Duh.

A serious hack attempt would have broken it, no question; there's no unbreakable DRM, after all, and I wasn't even using serious encryption (I decided it didn't matter: A typical attacker is not going to attack the encryption mathematically, they're going to decompile the binary to get the key AND algorithm, so as long as it can stand up to trivial attacks, it's strong enough -- connecting to a known encryption algorithm might actually make it EASIER for them, since then they'd have known function names and parameters). I just wanted to raise the bar.

Mere speculation: Isn't this just a result of the mixing of "Helpfulness" feature and the black-hat marketing / reviews purchasing?

As in now those selling "Reviews" for your app; something like $100 for around fifty 3 to 5 star reviews can justify their price by stating their reviewers have consistently ranked as being "Helpful" and always on the "front page" of any app review? And what we're seeing is the reviewers working for these black-hat intermediaries trying to up their overall review "weight"; whereby I'm assuming google takes into account your previous reviews (and their helpfulness) with how long your review now stays on the front page (much like comments staying on top of a HN thread).

Or it could just be a typical fad akin to wanting the personalized car registration of "A1", being top/first etc.

The sad fact is that most people are awful. I fully expect this trend to catch on elsewhere, it just happens to have got started off in Turkey.

When I read the title I thought: "well, must be an issue with internationalization (see Turkey test)". When I read the actual article I just thought "w..t..f... This just doesn't make any sense, who came up with this?".

They are just exploiting the flawed rating system. If users are encouraged by the system to give bad reviews I don't feel that they could be blamed. Google should think of other way to display review comments.

They could perhaps implement meta reviews, which let users rate reviews. Note That:

* perhaps the ratio of misleading ratings isn't necessarily high enough to render the system ineffective (iow, it may be just noise), but in order to know that, it appears to me that the meta review would be necessary.

* it wouldn't stop people from trying to game the system (which lead to the idea that even if meta review was there, its effectiveness would have to be tested, maybe by using... a meta meta review?)

Apple has done this for years. "is this comment helpful" and then the reviews can be sorted by most helpful in addition to recent. i'm pretty sure most helpful is the default as well.

I guess this wasn't too far fetched an idea. Google engineers must have thought of this before me as well, which means that they must've a good reason to not implement it. I wonder what it could be.

Abusing something is like a cultural thing in here. Sorry :(

Is pretty funny in a law of unintended consequences kind of a way.

I guess people should be able to rate comments. They would fight on comment rating and less on app rating. If the idiots are just a minority, this should fix it.

Perhaps you should only be allowed to give an app a 1 star review if you delete it.

Re-installing it should remove your review.

There's also a phenomenon in IMDB where Turkish movies get voted exceptionally high on many categories.

Turkish users like to manipulate votes and have some high self esteem to be a superior in doing anything. So if some thing is Turkish rooted and be or seems to be better than competitors they have pride. So to have more pride and fulfil their self esteem they try to manipulate every voting system.

In this case, the self esteem is their comment is better to bee seen then anyone else's so why not manipulate the system? It's some kind of hacking culture but in a bad motive.

And does this surprise you, op? I think you have seen ample evidence of so called "troll" comments garner more attention and clicks in ekşisözlük. They are doing this, because they want attention and google provides an easy way to do it. You don't even have to write a provocative content, just click 1 star and you are on the top of the world.

A community, be it a virtual or a real one, has its trolls, e.g. misusers. If you promote such a behavior, albeit unintentionally, increase in repetition of such behavior is very expectable. The thing to do in my humble opinion is rethinking this feedback mechanism and rebuild it in the content quality's favor.

this is simply the first shot in our generations new problem - how to gather review globally in a useful and fair manner.

"stars ratings" is simple - and in fact too simple. Rating based on likes of friends is the next (current?) evolution - but that has some terrible drawbacks for fairness and socially beneficial ends, after that? Probably based on actual usage, as most social sciences have given up on self-reporting as a methodology and started using observations.

But it's going to be a hard slog - and personally a slog I would prefer to be done openly and not behind closed commercial doors

Do 1 star comments really go to the top or is the problem here that these kids think they do? Isn't it more likely it's just the most recent reviews that are getting shown first?

1-star comments really go to the top...

Is this strictly limited to Turkish users? I have a feeling that if we were to consider apps in other languages we might find it's just a general problem amongst all users.

It could just be that the trend started amongst Turkish users first and the language barrier could be inhibiting its spread to other locales. This article having translated and explained the review ranking behaviour might speed things along.

This reminds me of the case where an Indian politician's facebook page was found to contain a lot of 'likes' from Turkish users.

http://www.ndtv.com/article/india/rajasthan-chief-minister-a...

The city where he was most liked changed from Jaipur to Istanbul which helped detect this.

They are mostly 9-14 years old kids. And yes, they are selfish, spoiled, senseless a bunch stupids. Do not think i am using "a bunch" phrase to de-emphasize this situation. There are 6.5 million 9-14 years old kids live in Turkey and i can guarantee that, half of them can do this shit. So google must do something before developers get more damage from this stupidity.

YouTube already realized that the 5-star rating system doesn't work [1]. Based on the histograms I see on the Play Store, it's seems like they should realize it too.

[1] http://youtube-global.blogspot.com/2009/09/five-stars-domina...

Another awful app review trend among Turkish users https://itunes.apple.com/tr/app/cloudsearch-reverse-phone/id... They give 5 star in order to get 3 free credits. (Also this app uses CIA app's database which is free)

To be fair, I'm not sure it has ever been a good idea to take advice from random strangers on the Internet.

Not sure that still applies as aptly in the emerging world where (nearly) everything is being advertised, explored, researched and ordered online.

Good or bad, the crowd is relying on its emerging "wisdom" more 'n more.

In America either something changes our lives or we want the creator to have their house taken away.

Turkish users "crack" reCaptcha http://radar.oreilly.com/2008/01/comments-back-on-recaptcha-...

Also the term "yorum" in Turkish means "comment". So it does not completely mean "review". In other words, there is no equivalent translation for the word "review".

"inceleme" may cover "review". it is used to be like this in turkish websites:

"Galaxy S3 incelemesi" "Samsung SyncMaster 2333 incelemesi"... etc.

Turkish users are used to the word "inceleme". Yorum/comment is like: ah yes I liked, fuck this sucks. A yorum/comment may not include why do you like it/why does it suck, etc.

I guess Turkish users will get blocked by app developers after looking at this ridiculous attitude. Btw they helped Google to find this bug, good job Turkish users.

It is not really a bug, 1 star reviews get an intentional bump in priority which is expected since "THIS APP BROKE MY PHONE, 1 STAR" is more important than "Just another breakout clone, meh. 3 star"

This is like : http://dilbert.com/strips/comic/2003-04-16/

Interesting. Any idea for another way "helpful comments" can be identified? I mean, presenting bad reviews is helpful in most cases.

As an iOS dev, you get a lot of crappy reviews based on really dumb, inane things by users.

People are very quick to give 1-star reviews without hesitation.

hi i am from turkey and i can say do not take seriously turkish reviews with 1 star. these are absolutely rubbish. some people in turkey just want other people to read their comments in anyway

i am also an android developer and i do not know why they are doing this shit.

IGNORE THEM.

I wonder if it is isolated to Google Play, and if so, if these are actually human accounts?

Better than "I would give it 5 stars if it was free" type of comments.

"I would rate these jeans 5 stars, but they were delivered a day late so I'm giving them 1 star."

That's nobody's business but the Turks.

I once did iPhone app development. I'll always remember one app in particular that I shipped. A retro game. We'd see lots of "reviews" in the App Store where the user would leave comments like, "great game! loved it. but I really wished there were more playable levels for the 99 cents I paid. So I could only give it 1 out of 5 stars."

ad nauseum

I've seen plenty of reviews along the lines of "This free app does everything it says it does, and does it extremely well. But it doesn't do <unfeasible thing that would matter only in some obscure use case> so until it does, 1 star."

Ha ha! It gave me a good laugh :-)

oops sorry, I thought it was funny that people can be so eager to have their comments appear but I missed the seriousness

This article seems naive. Probably people are being paid to rate apps down.

If they're being paid to rate the app down, why does the review itself claim the game is good?

Plausible deniability?

They are not being paid :) I am Turkish, and believe me every, but every apps' review section is full of these stupid 1-star reviews. Turkish apps, english apps, popular apps with ~5 million downloads, unknown apps with ~1000 downloads.. It is mind blowing. They don't know they are hurting developers and firms. Actually some of them thinking that they are helping with their "positive comments". They are showing their appreciation with a positive comment on the front page (with the help of giving 1 star rating).

What do you expect from Android users? In general the Google Play store is a far more bizarre and spammy atmosphere...

yah they should just work out some learning method to pick out stuff that looks odd & offer to clean it up. But i believe then it would require a thin layer of oversight and Google of course will not want to get involved. Apple clearly doesn't shy away from app store admin which is why they have the better platform at the moment.

Or they could of course just do away with the mechanism that auto-highlights 1-star reviews and instead you know.... ask the user what they want to see. As if people can't handle clicking a bar that says "Read 1 star reviews" a la Amazon.

I can't imagine why you would want to list 1 star reviews first... generally 1 and 5 star ones are the least informative (either "I'VE PLAYED FOR 5 MINUTES AND IT'S THE BEST GAME EVER" or "MY PHONE CRASHED WHILE DOWNLOADING THIS IT'S A VIRUS!!!", and 2-4 reflect though-out reviews.

Projecting much? Most mind bogling idiotic comment ever.

I don't think that Android users are any worse than IOS users however I strongly believe that Google Play needs to rethink it's policies.

"What do you expect from Android users?"

Sigh. What did i expect from a web designer? /s

> Sigh. What did i expect from a web designer? /s

I'm not a web designer...

"I design and develop websites at MasterMade or write at Newly Ancient."

From your website.

Well, he said 'or'.

Clearly more than we should expect from you.

Ah yes, the great unwashed and their non Apple operating systems.

Oh, this is glorious!

The reason I like is because your attitude (without your realizing perhaps) also typifies Apple users' stereotype.

Trying to execute a slick low blow to Android users, you have it explode in your face, and are instead reinforcing the stereotype associated with the Apple users.

(Whether it is true or not, doesn't matter, it is funny that you are reinforcing it).

TIL Android users are XXXXers

Reply on news.ycombinator.com