Hacker News Reader

Comment by arprocter | Hacker News Reader

A star rating is a bad way to measure - as a driver you wouldn't make the 4.6 grade mentioned if you got equal amounts of 4 and 5 ratings

Better to to ask 'would you ride/drive with this person again?' with a simple yes/no answer, and if no then ask for a reason - if people aren't given a reason why they got a lower rating then how are they expected to change their behavior?

The only time I wanted to give a bad review I couldn't - the driver left me waiting in the cold for 30 mins and then 'cancelled' without ever picking me up

View on HN

Put another way, star ratings are bad because different ratings mean different things to different people. To some people, like me, 5 stars means absolutely perfect/phenomenal/couldn't be better, while to others it means "it met my expectations", which I would consider 3 stars.

Amazon's star ratings are also pretty meaningless, for the same reason. And I used to bemoan YouTube's switch to a vote system from stars, then realised it was for the better.

Perhaps a better solution would be a three level system, with labelled letter grades: A (exceeded my expectations = perfect), B (met my expectations = good), and C (did not meet expectations = bad). Most people are also familiar with letter grades from school (at least N.Americans are), so people know exactly what each rating means. And the average rating could be displayed like "B+" or "A-", or "A+" for almost across the board perfect.

I don't think "exceeded my expectations" is even a useful rating since it creates the 4.6 situation all over again.

Not sure I even want an Uber experience that exceeds my expectations. I want one that is reliable and exactly what I expect.

A driver should be able to get a 100% rating providing the exact service that people imagine when they order an Uber: reliable, gets you from A to B, clean car, no problems.

With the "exceeded expectations" rating like the current system, some drivers bend over backwards in weird ways like by offering Snickers bars, mounting a TV on the back of their seat with a movie playing, and all sorts of random stuff that I just find jarring. Nobody should feel like they have to do that.

Well, maybe not for Uber, but I think it would work for Amazon.

The only star rating system I ever liked was Netflix, in the disc era. It worked in part because the stars had responses connected to them (“Loved,” “really liked,” “liked,” “disliked,” and “hated”) and in part because it was entirely calibrated to the user’s taste. It didn’t really matter what criteria anyone else used to rate a movie three stars versus five stars as long as you were consistent in your own ratings.

Ratings are way too arbitrary. I only give perfect scores in very rare situations. 5 stars to me means I can literally not think of anything that could be improved. I can always come up with a way something isn't perfect.

Do you want to revise your judging criteria, now that you have this new information that your giving drivers a non-5 score contributes to them losing one of their means of earning an income?

(not a rhetorical hit at you, just curious to know your thoughts)

I pretty much only give 5 stars even though I don't feel like that's what I should be giving. I was just stating if I was honest with my rating it would only hurt drivers. That's what I've always assumed and it turns out I was right.

I totally agree. 4.6 as a cutoff is insane.

I regularly rate things 3 if it's good, 4 if great (on Google Maps).

What Uber should have is:

Rate your ride:

5 - great

4 - bad

3 - really bad

2 - awful

1 - shouldn't be allowed to drive ever again

What I think they should do is just have one questions with two answers: “Would you ride with this driver again? Yes or no?”

That basically is what they have (see bottom):

https://www.uber.com/en-AU/blog/how-ratings-work/

I think it's absurd that they try to dictate user behavior based on broad averages, as opposed to normalizing each individual's ratings to fit a standard distribution, then averaging those normalized ratings.

All 5 star scales operate as follows:

5 - Good

4 - Terrible

3 - Terrible

2 - Terrible

1 - Terrible

That seems to have become the norm. To me that's just people misunderstanding how to use a gradual scale.

I recently got a phone call from my apartment management after I left a bunch of 3s and 4s on their annual review, they seemed distraught they got such bad scores. From my point of view they were doing just fine. The fact that they're not excellent in every regard shouldn't be surprising when I'm not paying through my nose to live there, I'm perfectly happy with 3.5/5 since that's what I'm paying for.

It's an ordinal scale with arbitrary meaning. 4 doesn't mean its twice as good as a 2 star rating (there is no interval or ratio). It basically means 5 = all okay! 4 and below there were issues of varying subjective degrees.

See also: https://xkcd.com/1098/

Ratings tend to be skewed in various ways everywhere. But it's particularly extreme when people know the norm is to give a perfect score for any transaction that didn't have serious flaws--especially when you know lower scores have material negative real-world implications for individuals trying to earn a living.

I think there's a reason Uber chose cutoffs around 4.6 (depending on the city) and I don't think it's because they consider someone with a 4.5 to be a terrible driver.

Uber faces a similar issue to taxis: the supply of potential drivers is too high. Too many drivers means each driver can't get enough fares. Cities solved this for taxis with the medallion system.

Uber could limit drivers arbitrarily by preventing new sign ups, but if their competitors don't do the same, this puts them at a disadvantage without solving the problem: the drivers they prevent from signing up will just go to Lyft. Using a star rating as a cutoff is smarter. The drivers with the highest star ratings are the ones the riders generally liked best. If Uber's competitors want to take the lower rated drivers that Uber rejected, that's fine with Uber. Riders will start to find, on average, that Uber has higher quality drivers, giving Uber an edge over competitors.

The 4.6 minimum rating may have been tuned so that there's enough drivers that riders aren't kept waiting, but not so many that each driver can't get enough fares.

Reply on news.ycombinator.com