Hacker News Reader

Comment by mox1 | Hacker News Reader

So if you follow the AirBnB host forums at all, anything less than a five star(absolute best possible) review is considered bad.

Hosts flip out all the time on 4-star reviews. They loose can "superhost" status and other such things after a couple non-5 star reviews. Giving a 1-star review is probably worse than causing $100+ in damage for some of a lot of these hosts.

Hosts will rarely rate guests < 5 stars, simply for fear of retaliation.

The problem here is that guests are never told that 5 stars = normal stay. This leads to problems. If AirBnB was more up front about expectations it would make their review system a lot better.

View on HN

And then when guests know this about the rating system, a lot of them will go even further to avoid giving a low rating because they don't want to harm the hosts' livelihoods unless something truly outrageously bad happened.

Just like a lot of people I know will tip 20% on a restaurant bill without even reflecting on the level of the service, because it's so ingrained into people that wait staff rely almost entirely on tips for their livelihoods so people feel guilty using this quality signal mechanism the way it was initially intended.

Because the way it was intended is inhumane if you even barely reflect on it

It's inhumane to give someone a bonus for good performance?

It is not a bonus, they are legally exempt from minimum wage laws.

The entirety of their livelihood is dependent on our judgements of their service.

If they don't make enough in tips to reach minimum wage, the employer still needs to close that gap. It's not as exploitative as we're led to believe, or at least no more so than working a minimum wage job is normally.

Yes but this never happens. If you cost your employer money at the end of the month because you didn't make minimum wage then you'll be fired. Employers don't want to pay out, employees don't want to be fired so nobody ever speaks of it when it happens and employees just accept less than minimum wage.

You are mistaken. Firstly asking your employer to close the gap is a great way to get fired. Secondly minimum wage is so low that you cannot possibly live on it. It's in fact so low that not even walmart pays minimum perhaps because the kind of people who would work for it aren't liable to be worth employing.

Federal minimum wage is 7.25 after taxes this is about 250 per week wherein most people in this situation are paid every 2 weeks. That is to say that one cannot expect to have more than 2 paychecks per calendar month for a grand total of $1000. If you didn't need food, clothes, medicine, or anything else this is about enough for rent.

What this means is that the employee/employer relationship not to mention the individuals ability to live indoors and have food is actually predicated on the ability to achieve greater than minimum wage. Tipping out even if they can do so isn't enough to keep anyone afloat. Not tipping in a state that has a tipped minimum wage is not that much different from dining and dashing its just not illegal because its never been illegal to screw someone if they are substantially poorer than you.

cactus2093 referred to how it was originally intended, not how it currently works. I believe the original intent was for workers to always be paid a decent wage by default, and to get a bonus for exceptional service. Now it's inhumane not to tip because tipping is built into salary expectations, but that's unrelated to the original intent.

it depends: that isn't true in san francisco, FYI.

The jobs aren't blanket exempt. That's a federal rule. A ton of states don't exempt tipped jobs from minimum wage. In California, for example, where it is the case, the usual minimum wage is $12 or $13, depending on the number of employees, and a dollar less for tipped jobs.

But repeating the pattern, that's a state rule. Different parts of the state don't exempt jobs from minimum wage. (God, I'm glad I'm not a lawyer). In San Francisco, for example, the minimum wage is $15.59, and as far as I can tell, it holds whether you're tipped or not.

Minimum wage didn't even exist when tipping started.

Is it not expected that when you pay money for a service then the performance should be good? Good service should be the default. If the service is outstanding then that is when we should start giving a bonus.

US tipping etiquette has nothing to do with good service and all to do with propping up underpaid staff.

I think we agree, it's more of a matter of adjective grade. I expect service to be adequate. Some of it is frankly good. Once in a while it's outstanding.

As for US tipping etiquette, cactus2093's point was that current US tipping doesn't really reflect what was originally intended. But kingkawn made a claim that extended past the current US tipping practices.

The argument that you should always tip food service ~20% is generally based on the claim that "quality signal mechanism" is not actually the intent of the tipping system, but merely an excuse (that was perhaps true at some point in the past) to justify legally paying workers low wages and artificially reducing the sticker price of food.

Not only do they flip out (which means you leave a 5 star review to avoid drama) but if you have a really crappy host that cancels your reservation or run into any weird issues you are not ALLOWED to leave a review. Therefore the only reviews you are seeing are "successful" stays. I'm never going back to AirBnB. It's been hot garbage all the way down. Can't trust a thing on there.

An automated review is posted when the host cancels, indicating how far out the cancelation from the stay happened.

Do these reviews include stars? Ie would be good to know how the cancellation was handled. There was a recent example in NZ where a host cancelled on a family based on their ethnicity (Covid-19 fear related), would the automatic reviews provide insight into such issues with hosts?

The current system can be improved, certainly.

I've mostly ran into issues where the host is not able to work out a significant issue (unresponsive for days, does not provide access to property, property smells so badly of smoke you cannot breathe adequately - as a former smoker this is a high threshold). This is not technically a cancellation but an "ended stay" and you are not able to leave a review once you have entered the property in my experience. That said, I'm sure it's improved (hopefully in the year+ I haven't touched it)

In general, it's been extremely difficult to read between the lines on reviews since they're all five stars, and sometimes you end up reading things unintentionally.

> The problem here is that guests are never told that 5 stars = normal stay. This leads to problems. If AirBnB was more up front about expectations it would make their review system a lot better.

They’re not entirely explicit about it, but to leave anything less than 5 stars AirBnB you have to leave a text explanation of why you did. So it becomes clear quickly that 5 is the norm and less is for an actual problem.

If you score around three stars each time then you’ll see the prompt each time and not think anything of it.

Yes, this is such an anti-pattern.

Once Uber started asking me a thousand questions about why I was leaving someone 4-stars, I stopped leaving reviews entirely.

Lyft does the same. And when you tap on 3-stars it even says "Just OK" yet it treats it like it was a horrible ride.

If a Californian told me a restaurant they went to was "Just OK", I would avoid going to that restaurant unless there were no other alternatives. I think that might be part of the disconnect.

That is something that I find weird in US "ratings". To me, "ok" means that it was... Ok. Definitely nothing bad, from completely satisfactory to good, but not exceptional. Which for some reason seems to translate in US to something like "Awesome, best food ever in my life". Now, my question is, what do you say when you actually get awesome food that is best you have ever eaten?

You just say the same, but louder.

"OK" and "Just OK" are two different things. Californian culture tends to shy away from direct conflict unless it's deliberate so there's a lot of damning things with faint praise.

Speaking of subtle differences, what's your take on "A-OK"? That's Yelp's label for 3 Stars.

I also think that it's different for restaurants than for drivers. When looking for restaurants whether in my home city or traveling, of course we look at 4 stars or above. But never have I once looked a driver's rating, I just call a car and take the first one that is assigned to me, cancelling only for reasons not related to ratings at all. Maybe it's just me that does that?

> Today the average rating on Yelp sits in between “A-OK.” (3) and “Yay! I’m a fan.” (4).

src: https://blog.yelp.com/2018/09/restaurant-ratings-on-yelp-are...

Has that actually happened? I’ve lived in the US for basically my whole life and I’ve never met someone who disingenuously described a mediocre eating experience as the best in their life. I think it would be hard for them to make friends if they can’t even be honest with how they feel about meals.

I think that depends on where you live. I’ve lived in places where there were not that many restaurants better than “just ok” unless you wanted to spend a lot of money. Nowadays, I feel as you said, that I never have a reason to go out of my way to new place that’s “just ok” but that’s because I live in LA (and previously SF) where there are “seriously great” options everywhere. My standard for affordable food has gone way up after living in California.

My dad always says "Yeah, it was a pretty average house. Nothing terrible, nothing amazing, 2.5 stars". He then clicks in the middle of the middle star, and ends up giving 3 stars.

Hosts must hate him.

I wonder how changing the rating system in cases like these from a five star scale to a thumbs up/down system would affect user behavior and system usefulness. It seems like ebay, uber, airbnb, etc all suffer from the same "anything less than 5 stars is effectively negative" issue.

Once you have a large enough userbase the thumbs up / thumbs down method might be the best option. My favorite example of numbered rating systems failing is metacritic where almost every user score is either a 10 or a 0. They have a weird dynamic where users will give a 10 to counteract a low score or a 0 to counteract an inflated score. Giving a 0 or a 10 makes your vote more impactful so why would you score any different?

At the end of the day most people just want to know if people think a service, movie, driver, seller, etc. is worth it or not * . If you need more info you are going to read the reviews anyway.

* An exception might be niche user review communities like myanimelist or rateyourmusic

Who looks at the user reviews on metacritic?

Lots of people?

The difference between the critic rating and the user rating is useful information to have when making a viewing decision. If they are both low, that tells you something. If one is low and one is high, that tells you something.

Grading scales can work, for instance if people are rating and reviewing products in online shops.

You're looking at a jacket and you see that while some people rate it 5/5, most of the ratings are 4/5 and 3/5, and 85% of the reviewers say they would recommend the jacket to a friend.

So you read the reviews, and gather that most of the medium scores are because the arms are a little shorter than expected and the cut across the back is rather wide. You also notice that everyone is praising the quality of the fabric and worksmanship. Good information, since if you have short arms and the back muscles of a powerlifter, that jacket would probably be a good purchase.

That is good use of a scoring system, but it requires people to care about it and about writing reviews. It works for niche sites, but completely breaks down somewhere like Amazon, or on Uber/Airbnb where you get punished for anything less than a perfect score.

Ebay has a positive/neutral/negative review system for sellers, right? Does that have any interesting side effects?

This reminds me of the idea of net promoter score in customer satisfaction surveys. It's a variation on the idea that it's better to have imprecise knowledge that is relevant than precise knowledge that is irrelevant. Net Promoter Score replaces a litany of questions about satisfaction with a single question ('would you recommend this product to a friend' (yes/neutral/no -> 1/0/-1)? You take the average of this score ((yes - no) / total) and it corresponds roughly to the health of a product. The basic idea is that asking a bunch of questions doesn't really add much knowledge, and giving more choices and asking more questions can actually lead you down a false path.

Everyone who has ratings does it. I used to rate 3 = professional and fully acceptable until I got a call from Dell asking what they had done wrong, and then from Honda. I realized ratings are all inflated and staff are effectively penalized for less than perfect, and since I'm not going to screw someone because I disagree with their rating scheme, I just set 5 to acceptable and go off that.

The sane way to do it would be to ask yes no questions, but that would require thinking.

So why leave ratings?

Nearly every time I have a bad experience with a company the problem is systemic, but a bad rating will be associated with the random person who served me. So I don't participate. If the problem is particularly bad I find another supplier.

If they did their job, I'll give them the "you did your job" 5 stars.

Even if the company has systemic problems, they still need to pay bills as long as they still have the job.

Finding another company is a smart option. I'd also write to the management to explain why that problem is blocking you from doing business with them, without mentioning who you spoke with.

I feel a better system is just thumbs up/thumbs down. Either it was satisfactory, or it wasn't.

Basically the same as our current 5 star/1 star dichotomy but it makes things more clear to people who are tempted to give 4 stars.

Even though it seems like shrinking the available options will lead to less fine-grained data, sometimes it seems like it actually works out better. I really liked how Four Square only had thumbs up and thumbs down as the allowed ratings. I feel like it did a very good job of capturing how aligned with expectations the restaurant experience was. It also made it really painless to give feedback.

Yup, I really miss UrbanSpoon for the same reason (now owned by zomato and totally ruined). I found it's percent-thumbs-up metric to be so much more valuable than Yelp's 5-star average. If you really want a finer-grained system, have multiple thumbs-up, thumbs-downs.

YouTube originally had a five star rating system for each video but switched to the current thumbs up/down system. I believe the reason given was that almost all viewers either give five stars or one star, which made the middle three ratings pointless.

How do we make a single thumbs down not trigger a 5 alarm fire in the customer service department? Anyone else dread the onslaught of follow up from a company after you give a bad rating?

Is there a dichotomy? I don't see the 1 star being more common than 2 or 3.

I changed my music player/library from the default 5-star rating system to a 1-star system, basically replicating the "like"/"save" functionality from Spotify. It makes it easier to handle, because I don't have to think about whether I like this particular song 4 stars or the full 5 stars.

I just mark favorite tracks and favorite albums. The track and album ratings are separate, because I might love an album as a whole, but also have a couple of individual favorite tracks on that album.

And I don't have any music in my collection that I don't like or am completely "meh" about. If it's in my library, I already love it, the question is whether it's an all-time favorite or not.

TFA's thesis is that people convey negative information -- just not primarily through the star rating.

As a longtime Airbnb user I can't imagine staying at a place just because it has a good star rating. You have to read the reviews and see what people liked and didn't like. If a place has a bad star rating I generally wouldn't consider staying there at all, but a 4.8 star rating with lots of substantive reviews and no real complaints is very different from a 4.8 star rating with a bunch of people noting that there's deafening construction noise from 8AM to 8PM.

Also, how would anyone retaliate on Airbnb? You can't see the host's review until after you've posted your own.

How would retaliation work? I don't think I've ever been able to see my hosts review of me before submitting mine for them.

> If AirBnB was more up front about expectations it would make their review system a lot better.

I guess from a strategic point of view it might look better for the AirBnB's and Ubers of this world to have lots of '4.7 out of 5' ratings than 'xx% of people wanted the service provider to be punished', especially to first-time users. But yeah, if Uber in particular wanted to honestly represent what the ratings actually represented it'd be a simple upvote/downvote.

Ironically, on platforms more focused on allowing consumers to choose based on qualitative reviews than penalising based on quantitative scores, I tend to find 4* reviews are much more likely to be well-informed enough to influence me into buying the product than the 5* ones, as well as more likely to be a genuine reflection of the purchaser's sentiment.

This seems to be a problem with most similar review systems. I believe Uber has similar issues with anything less than 5 stars being considered unacceptable and causing problems for drivers.

I used to give most Uber drivers three stars until someone told me it’s not supposed to be like that. There’s nothing in the app to guide you that the scores are shifted like this.

> If AirBnB was more up front about expectations it would make their review system a lot better.

Better for consumers but not better for AirBnB. The fact that potential customers assume that 5-stars indicates an exceptional rather than standard review is an advantage for AirBnB. Revealing the actual rating expectations would cheapen the perception of all the existing reviews.

There is a real sense of rating inflation happening here. It makes a 4-starred place seem especially sub-standard, even though it might be the result of a single unreasonable guest, or even a reasonable guest providing a muted review.

Yelp is hated in the restaurant industry for similar reasons.

Do users really evaluate like that?

For me, it's pretty obvious that rating is arbitrary and depends on how optimistic/perfectionist/obsessive/etc. the reviewer is and that there is not much difference between 4 or 5 stars.

It's not users, it's the platform itself. AirBnB and other companies like Uber and Lyft will significantly deprioritize you if your average rating isn't very close to 5 stars. Rating your Uber driver 3 stars can significantly affect how many fares they're offered and therefore how much money they can make.

But if users don't care, why do platforms care? Maybe it's actually a reliable signal?

Obviously it's also more reliable in the context of other evaluations by the same reviewer which the platform has easy access to, i.e. if someone always gives 5 stars, then a 4 star review from him is much worse than one from someone who never gives 5 stars.

Users caring might mean that a user is 15% more likely to select a 5 star host over a 4.5 star host, not that the user will say a 5 star rating is a top concern.

They could circumvent that problem by replacing the five stars with an up/down system -- since that's the way it's interpreted anyway. I don't think most users are capable of giving more nuanced reviews anyway. If they can, it would be better expressed in words than as numbers.

If I have a stay that's genuinely superior, I'll write an enthusiastic review. Other than that, "it was fine" or "don't come here" is really all I need to say.

Wouldn't a binary rating system work a lot better here? Either positive or negative + a comment, sounds like it would work way better.

> Either positive or negative + a comment, sounds like it would work way better.

Exactly. This is how CouchSurfing works. It's just another way of asking "Would you recommend this place / this person to your friend?". And that's what people really want to know, after all.

The fundamental problem is that as soon as a metric is used for financial incentives, it becomes useless as a metric.

Great so now I feel bad for all those less than 5 star reviews I left.

Sounds like eBay, buyers can't get anything other than a positive review. You can write whatever you'd like in that review but no one reads them.

It sounds like they should switch to a binary review system (thumbs up/down). Leave the nuance to the actual review text.

sounds uber and airbnb should ditch the 5 star system and switch to thumbs up/down like netflix.

With Netflix the 5 star system made sense and they shouldn't have ditched it. There was no issue of someone losing their livelihood if you left a 4-star review on Netflix, so Netflix movie ratings were pretty accurate I think.

It's the same thing with Uber and Lyft. Dropping below 5 stars can result in termination.

Reply on news.ycombinator.com