Judge: There’s no proof Yelp manipulates reviews | Hacker News Reader

Judge: There’s no proof Yelp manipulates reviews

arstechnica.com

109 pointspavornyoh109 comments

The standards for adequately pleading a securities fraud lawsuit -- which this was -- are incredibly high, and the plaintiff has to meet that standard without access to any discovery from the Defendant (here, Yelp). Which is all a way of saying that it is extremely difficult to read too much into the dismissal of a securities fraud lawsuit.

The judge in this case was basically concluding that plaintiff -- a Yelp investor -- could not adequately allege in his complaint that Yelp or its executives made a false statement of material fact in connection with the sale of a security. And plaintiff had an extremely high burden to survive such a motion to dismiss because of the Private Securities Litigation Reform Act (https://en.wikipedia.org/wiki/Private_Securities_Litigation_...).

So, in short, there is very little that one can conclude about Yelp's actual practices (for better or for worse) from learning that a securities lawsuit like this was dismissed.

Yelp as a company has no way of knowing what shady practices their sales people use to try and get an ad sale. I've experienced first hand being the point of communication for my parents' small business. As soon as I declined a $300/month offer, my parents' business (which had only 2 5 star reviews) suddenly received a 1 star review from some random person that didn't even fit the store's clientele. Where did this review come from?

I understand the benefit Yelp provides to some excellent establishments but it unjustly hurts so many along the way.

I've been hearing these stories too. But till date I've yet to find one (just one!) piece of incontrovertible evidence that Yelp sales folk engage in blackmail. Surely, after all these years, there must exist just ONE phone recording or email??

I'm convinced that there is an untold number of third parties/scammers cold-calling listed business numbers falsely representing themselves as Yelp, targeting owners who are not technologically savvy.

Unfortunately, Yelp's current model seems to encourage extortion at the hands of these third parties, and is also incapable of punishing them.

What I think happens / Paraphrase of what I'd imagine a 3rd party call to be like:

"Hi, this is Yelp. Pay us and we'll boost your rating/visibility and hide bad reviews. If you don't pay, your Yelp ratings will suffer."

Then they just manipulate ratings and review visibility with fake accounts. Add good reviews and hide bad ones if the owner paid, and add bad reviews and hide good ones if they didn't.

(Also I'm aware that there are plenty of "legitimate" businesses that provide this service and don't misrepresent themselves as Yelp itself)

And you should be convinced.

I'm a Google Trusted Photographer in Houston and I visit and talk with businesses that get targeted every single day. I'm loosely affiliated with Google and I have to make that clear. They get calls twice a day from "Google" and "Yelp" about their pages and reviews. This is their #1 complaint when I call them or walk in their door. Being local helps.

I've gotten a few prerecorded calls from "Google Local" about my business, saying something about needing to verify details. I never really listen though, because I don't have a business (they're calling my personal cell phone). It's really quite strange.

Try having a listing on Google maps and see how many times you get called.

You know, the old protection rackets (which probably still exist) were easy to make seem non-illegal. If that's a word. One person offers to prevent the consequences of ignoring threats, but doesn't make the threat themselves. A person with little or no known connections to the first makes the actual threat and carries them out. The first guy was trying to help a neighborhood business with a potential problem, you see. In other words, if you pay me I'll protect your store from these lawless thugs destroying other people's stores who do not pay for my protection.

Now, looking at such things from a common sense perspective makes it quite obvious what's going on. But proving the same in a court of law, which often will have a much higher requirement of proof, may not be so easy.

You didn't look very hard: https://www.reddit.com/r/videos/comments/3k2bxt/yelp_salespe...

Rossman embellishes his stories. He once tried to claim that the recent debacle with error:53 and the iPhone was a conspiracy by Apple to brick phones and increase sales.

I wouldn't believe him if he told me it was pouring rain outside. I would check first and make sure it wasn't just a drizzle.

They don't need to say anything incriminating to execute this.

If you decline, the Yelp sales person could simply have their friend visit your business, have a "1-star experience" and then write a review.

Hey they faked the moon landing they can do this, surely.

Seriously though, I'd agree: you'd think at least one of these shady sales people would fess up for the publicity alone.

> Yelp as a company has no way of knowing what shady practices their sales people use to try and get an ad sale

Sure they do, it's just too convenient to look the other way and maintain deniability.

> Yelp as a company has no way of knowing what shady practices their sales people use to try and get an ad sale.

This phone call may be recorded for training or quality control purposes.

Right but how long do they retain any recordings and what percentage are reviewed for policy violation? And what if the employee calls from an unmonitored phone (personal) on their own time or even company time in order to increase their personal sales at the expense of company reputation, for example. It's not unheard of for sales staff to go off the reservation in order to better compete against hamstrung staff.

That's a matter of making sure that any sales representatives that might get ideas like these realize that such are capital offenses, as in: do this and you're out.

Which has happened. The Rossman incident shows that reps are fired for breaking rules regarding do not call and do not email and for having friends write false reviews.

I've also seen this first hand, except rather than a bad review, a bunch of 5 star reviews were made invisible. 3.5 stars on yelp vs 4.8 on google. From then on I've always used google ratings instead of yelp, which also has the benefit of being integrated well into the maps app.

Yelp reviews contain more content than any other service, whereas most google reviews I read that have anything more than a star rating come from illiterate or barely-literate foreign reviewers. My wife had a time period when every one of her reviews on yelp was flagged even though she put a ton of effort into her writing, whereas I never had a problem. Their spam/business-owner detection system catches a lot of false-positives, but that doesn't make the ones that do make it through necessarily worse. Verifying an email address and linking with facebook (although not necessary), and having friends helps reviews on yelp not get auto-flagged.

I usually dive into reviews and see what people are saying, not just go by a star rating. It adds to the hilarity when you read a 1-star review for a restaurant because they "are scumbags who serve lion" (actually, a menu item had dandelions on the side) or because someone drove by and saw a balkan restaurant "roasting a dog on a stick" (actually, it was lamb, but I had a great laugh).

Facebook should not be the basis of a web of trust.

I don't disagree. I also think their automatic flagging system is too aggressive and opaque partly because of such things having influence.

Google ratings are much easier to game, from what I hear

Blackmail notwithstanding, what was the $300/month meant to be for? How do they rationalise it?

You read a LOT of Yelp blackmail stories (or I have). But it isn't clear what legitimate product they're actually selling to businesses, since they'd at least need a cover.

PS - I am surprised any business has less than five reviews. If I ran a small business, I would create fake reviews just to get the ball rolling, no matter how immoral that is. Just makes good business sense.

Advertising. You buy, for example, searches in your category in your zip code(s). It's pretty excellent advertising.

Network effect. The important metric isn't how many reviews you have, but how many your competitors and neighbors have. If no one uses yelp locally, nobody wastes time on it. My daughter's optometrist has no reviews and the most popular optometrist in the county has five. Clearly a waste of time. My favorite "old time" hardware store has one review. Yelp seems to be for coffee and food. Observation shows possibly as many as 0.1% of coffee drinkers will write a review, and a local family run very popular burger joint that all the non-vegetarian locals have probably eaten at has over 0.03% of their visits reviewed. That is far higher than doctor rates or hardware store review rates. I drove past that restaurant yesterday and they had more local license plate cars in their parking lot than total lifetime yelp reviews... Clearly whatever it is they're spending their time on is more financially rewarding than boosting their yelp score, LOL.

They sell advertising on their own site.

If someone is looking at coffee shop A, they would show ads for my coffee shop.

It's excellent until you realize that you're paying WAY more for every click than you would with AdWords and that their reach is only limited to Yelp and doesn't do THAT much to boost your presence. My mom was considering it for her daycare business but I argued against it for this reason and many other small businesses that have dealt with them concluded the same thing.

yup, same here. I help SMBs with marketing, and Yelp is usually never worth paying for.

The only small case is when the ROI on adwords is reaching the level of diminishing returns

So does Google :)

Except for recording phone calls, monitoring emails, etc. Which is what they bring to court. Which is why none of the lawsuits go anywhere.

In reality it could be a third-party business that is wanting to then upsell you in the future positive reviews to counter-balance.

Yelp will perish, because, unfortunately, the moral quality of the individuals isn't high and the founders/CEO seem to be blind to it.

As much as you need to be a very good business man to build a business that Yelp is, you need to surround yourself with good people, and recently, plenty of good people have left Yelp, and realistically, there aren't many out there.

I think this type of thing will be the eventual death of most social media type enterprises. People just don't realize it's hard to maintain and curate such things, worse if it gets popular. It's especially difficult when so many people just don't understand how the internet works and believe every trollish thing they read on it.

We've witnessed this manipulation first hand. My son used to go to the local day care that had stellar reviews on Yelp - 4 or 5 stars with a single 3 star review. I knew the vast majority of reviewers because they were parents whom I've met through kids birthday parties and the like. Then one day Yelp called the owner of the daycare asking her if she'd like to purchase some advertising. She already had plenty of business so she politely declined. A week later 20-something reviews went into "Not Recommended" category, effectively becoming hidden. The only review that remained on the site for the business was the single 3-star review. This is effectively extortion. In other words yelp won't fabricate reviews, it'll just selectively filter reviews if you don't pay them to pain a worse picture of your business.

I have a small anecdotal experience with yelp manipulations. In my case as a user trying to search for a restaurants reviews by the exact name of the place. The search resulted in a competing restaurant blocks away of the same ethnicity that happens to pay for yelp advertising. This is pretty poor practice. The only way to find the restaurant I was actually looking for was with the map view.

If the excuse is poor search quality, it seems too convenient.

Literally every local small business I know reports some type of shenanigans with Yelp. Some of it is surely false-correlation by emotional shop owners.

But there is, at best, a critical mass of unethical salespeople, because I've heard far too many reports of pressured selling.

Either way, they've got a really big problem. Big picture, it's hard to imagine how a site like this can remain neutral or useful long-term. It's just a playground for mischief on all sides, and eventually consumers will catch on.

Fun Yelp story time:

Dated a girl. Her father was an eye doctor in a small town. There were 2 eye doctors on main street. He was convinced the other doctor had someone leave a bad review (which out of his ~8 or so reviews) which made him a 2 star "offering".

He was my eye doctor and I _did_ like his work, so we made an account, and left a review. My review was "untrusted" because my account didn't have any other activity.

Now it became a game. Everywhere we went we'd leave a yelp review to try and get my account to be "trusted". This went on for about 3 months and we had about 10-15 reviews. My account was still untrusted. We decided to stop and just let it be. About 3 months after that, my user was trusted and he now has a 5 star rating.

The woes of yelp in a small town.

I'm missing how this story reflects poorly on Yelp. You did exactly what they want Yelpers to do: Review a series of places so they can (after a few months, apparently) calibrate your ratings and have some algorithmic confidence you're not a drive-by or paid reviewer.

I think he's saying that Yelp is bad because he should not have been trusted. The whole point of the exercise was for his girlfriend to write a review for her father's business. But it seems nearly impossible to stop people who are determined to write fake reviews. Yelp obviously isn't perfect, but what's the alternative?

>I think he's saying that Yelp is bad because he should not have been trusted.

Based on what? Magical intuition that the account belongs to the boyfriend of the business owner?

"what's the alternative?"

Live a post-online-review existence. That's where I'm at. The future is here just unevenly distributed.

Take all the arguments people have against IQ as a measure of a person's meaning or value, turn them around and apply them to the number of online stars a company has purchased, err, uh, obtained via reviews from real customers LOL as if anyone believes that's really how it works. Or if you work for a big corporation you likely have extensive experience with people freaking out about utterly meaningless metrics... like number of online stars.

Sucks to be an online review company, but if the providers hate the implied, rumored, or actual extortion racket, and the users find the data to be useless, when they go away, nothing of value will be lost.

It's not uncommon for big community websites like Yelp to distrust brand new accounts. It sounds like Yelp could improve their "trust" algorithm, but this doesn't really surprise me.

There's always bias of various sorts though even if it's just the preferences of most of the people doing reviews. I liked Zagat in the pre-Google days, for example, because the demographic of the people willing to fill out a dead-tree mail-in survey about restaurants was probably pretty similar to me. (I still prefer Zagat to Yelp but it doesn't have as broad coverage.)

For all its various biases, I find Yelp OK given some critical mass of reviews. And reading some reviews can be as useful as looking at the overall rating. It's not great but, in an area I don't know, it's better than throwing a dart at the wall.

There is a two-week span every couple months where a Yelp rep tries to contact me daily, sometimes even multiple times per day. I rarely take unsolicited calls since I tend to get a lot of them, so I'm not really sure what they want. What do they have to sell you on? The company I work for doesn't have a strong "untrusted" Yelp rating, but it's largely because it's not a local business and people predominantly give us bad reviews when something negative happens, even when it's outside of our control. There are better review sites in the niche we operate in and there are no parallels between those reviews and what ends up on Yelp. You get some of the same complaints, but there are also far more positive reviews and the reviews are higher quality and more informative than what's on Yelp. For trusted reviews on Yelp, we rate much better.

They sell ads, including those that show up on your competitor's business page.

What does a useful local business review site look like? Not being snarky--genuinely interested to see if there's an opportunity here.

I think Yelp is missing proof of identity and proof of patronage. It was created before social media took off, and before you could reliably know if someone had been somewhere.

I don't know if you need both, but proof of patronage alone might not be enough to trust a review (See Amazon verified purchases).

Aren't Yelp "Check-ins" factored into the trustworthiness rating of a review, or whatever it's called?

That's kind of an ersatz proof of patronage.

Maybe. It still wouldn't be foolproof - after all, Yelp Check-ins are powered by an API that trusts the coordinates it receives.

It might be possible to craft a request for that API. And even using a legitimate copy of the Yelp app to make requests, a jailbroken iPhone can change its coordinates to whatever you want. An Android doesn't even need that - just set it to developer mode and you're ready to fake check-ins all day.

Would you ever trust an app that was linked to your identity and stored a list of places you visited? Be careful what you wish for.

Is that much different than what Facebook or Google Maps are already doing behind the scenes? They might not make that data accessible to you, but it's undoubtedly being (or able to be) compiled behind the scenes. See this update:

http://techcrunch.com/2015/07/28/google-search-now-shows-you...

The average HN user might not care for such a system, but I think many people would find it reassuring for better or worse.

It's a great question. I think a lot of the problem is reviews get stale AND you don't know/trust the reviewer.

Yeah. I don't live in the boonies but I also don't live in a big city. Pretty much any restaurant or small business around where I live has, at best, a handful of reviews over the past few years. It's just not very useful.

Biases baked into the platform are one problem for a review site. But you also just need critical mass.

I think the biggest things working against Yelp are the reports from businesses that Yelp offered to censor bad reviews and/or promote/shill positive reviews in exchange for money.

I've heard about it all N-th hand, so I'd imagine that Yelp's team is very careful about wording it such that it's some sort of legit promotion that they're offering.

...and yet, despite what must be hundreds of thousands of calls or emails made by Yelp sales folk to businesses, nobody, to my knowledge, has ever produced a voicemail, telephone recording, or message containing a promise or threat to do anything other than what Yelp publicly offers to its paying customers[1].

I'm actually sure this has happened, since it's essentially impossible to prevent thousands of salespeople from deviating from an approved script when pushing hard for a sale, but surely it can't be commonplace.

[1] https://biz.yelp.com/support/full_service_advertising

Exactly. I made the same comment above before reading yours. I also find it hard to believe that after all these years, not ONE person has solid proof of shenanigans.

Zagat?

Zagat is (still) pretty decent if you're interested in restaurants in one of the metropolitan areas that it covers and you're in their foddie-ish demographic. However, I don't think Zagat, which had a pretty long pre-Web history in what we now call crowdsourced restaurant reviews, offers a lot of insight for how we might do a small business review site today.

The question didn't qualify "useful" as being crowdsourced or not. And that gets to my point, a review site that has its own reviewers is pretty much the only thing that is going to be useful. Otherwise the system will be gamed by vendetta seekers and the business itself. And I believe it is inevitable that a crowdsourced site will ultimately end up gaming the system to try to offset those effects.

The bigger problem with crowdsourcing is that the reviews doesn't have a calibrated baseline. As a business traveller at 150+nights/year, my demands on a hotel are different from a leisure traveller's. Having a nightclub in the hotel is cool if you're going to that nightclub, but it's a special kind of hell to get back from a long day and have to fight your way through a nightclub line to get into your $600/night hotel (never mind having to deal with the noise from the nightclub). Another pet peeve of mine is to deal with clever/funky/cool "concepts", like that one time I tried to order a drink in a all-organic bar (according to the bartender things dry and bitter can't be sourced organically, so they only did sweet drinks).

Trip Advisor (most relevant for hotels) and Yelp is supremely bad for surfacing the aspects I care about from the depths of things I don't. It's better with restaurants, as most people seem to agree to a much higher degree on the baseline.

>pretty much the only thing that is going to be useful

IMO that's an overstatement. I find Zagat, Yelp, and Amazon reviews all useful--however imperfect. That said, however unpopular this opinion may be in our crowdsourced world, professional reviews/ratings tend to be more reliable than random ones on the Internet. The problem is that professional reviews don't scale, especially for local businesses. The New York Times restaurant reviews aren't bad but that's NYC and, even in that case, a lot of the reviews are going to be quite old.

So, in an ideal world, there might be armies of professional evaluators of hotels, restaurants, and other businesses--but that's not the world that we live in. (Films are one of the few areas where you can get a pretty good pool of professional and semi-professional reviewers.)

I think smart pay systems might help here. I'm not an avid reviewer - I just don't care enough for it to be worth my time. However, uber makes it part of the workflow. To get another ride I either need to make a rating or skip making a rating - so I make a rating. And it only takes one click. I suspect if Apple Pay or Google's equivalent had a similar process they would be able to generate some very sold scores for local businesses since they have both identify and proof of purchase.

Yeah but with Uber, the expectation isn't that you're going to leave a nuanced review about the comfort of the car seat and the smoothness of the ride, but to basically leave 5 stars so long as there was nothing horribly wrong. Which is fine. It's more or less a commodity service.

With restaurants, on the other hand, there is a wide range of quality. Even if the average shopping mall restaurant delivers a hot meal in a reasonable timeframe, I'm probably not going to give it 5 stars.

If reports are to be believed, then clearly Uber expected that it would be 5 stars or nothing, and their low tolerance for 4 stars makes reality more consistent with that belief. But as a user from the early days, nothing in the app clued me into this expectation and accordingly I gave 3 stars for "meets expectations", 4 stars when I was offered refreshments and USB connections, 5 stars when I couldn't imagine a better ride...

It's only lately and due to the prevailing rumor of Uber's mercilessness that I give 5 stars to meet "don't fire the driver on my account" and less than 5 stars to mean "I demand a refund and the head of the driver."

Back in the day when it was primarily an "Internet flea market", although it didn't use a star system, eBay was similar. Both sellers and buyers expected positive feedback accompanied with a comment like "A++++++++++ turned water into wine" so long as the reality wasn't "sent me a box full of bricks and axe murdered my extended family." Neutrals for a transaction that didn't quite go right even if issues were resolved were NOT appreciated.

Yelp is all but meaningless in San Francisco, probably its most popular city. Really average or even crappy cafes, salons, restaurants have greater than or equal 4 star ratings... it's a little baffling. I presume from the stories you hear that it's because the businesses pay.

This may be the case in San Francisco. It's not the case elsewhere, however, at least for this anecdotal sample size of one. :)

Business travel takes me all over the country and Yelp is incredibly reliable. I've lost track of the number of times that someone recommends a place, Yelp says it's a 2-3 star place, we go anyway, and yup, it's average at best.

I hear and see the complaints of restauranteurs about Yelp. Yet my experiences at places usually match what's seen in reviews. There's a restaurant in my hometown where the owner, a friend of mine, constantly complained about Yelp reviews where the the sentiment was that his food was too expensive and not of consistent quality (I agreed with the reviews). A few weeks ago he decided to close the place because of declining revenues. Had he listened to the reviews, settled for a bit less margin, and worked with his staff to produce a consistent product he'd still be in business.

Edit: I neglected to add that it's really important to read reviews for context. Sometimes you just have people bitching because they're a gluten-sensitive nut-allergic vegan and the place didn't cater to their highly specific needs.

I too travel all around the country for business, and I find Yelp to be less than helpful. I agree with you that reading the reviews for context is critical. In fact, that's the only way to really get a read on a place. But with hundreds of restaurants to choose from, and hundreds of reviews each (if there are only a handful of reviews, you can't rely on them at all), it's incredibly time consuming to go through them. I find that the star ratings are pretty much worthless. They get dragged down by people who ding fine dining restaurants for not having Olive Garden prices, and others who mark down local greasy spoons for not having French Laundry service. I don't care how the diner compares to the French Laundry. I care how it compares to other diners. A single rating scale across all types of restaurants is probably counterproductive, but the problem is greatly magnified by throngs of clueless reviewers who have bogus expectations and are rating places based on them.

Collaborative filtering is the obvious way to make a scalable restaurant ratings service. Google tried to do it and apparently failed; then Yelp tried to do it [http://officialblog.yelp.com/2013/06/whats-nearby.html]. Not being a Yelp user myself, I don't know if they are still doing this. Anybody have insight into whether Yelp is still personalizing recommendations or why personalization hasn't worked in restaurants (in contrast to e.g., Netflix)?

IMO, the star rating is useless as 4 stars can often be worse than 2.5 stars, but some random reviews can be helpful.

So, if your willing to spend a while going over them it's better than nothing. However, actual restaurant reviews are more accurate, take less time to use, and your less likely to read 10 reviews and learn nothing. IMO, Yelp way over hyped for how little value it's actually providing.

This may be the case in San Francisco. It's not the case elsewhere, however, at least for this anecdotal sample size of one. :)

Could we stop shutting down opinions using the sample size argument? It's one of the weakest arguments you can make:

'I think different so your opinion isn't valid because anecdote!' 'Oh you don't agree with my opinion? Well that's not valid either because anecdote!'

Not every discussion is capable of having prior polling as evidence (polling has its own issues with bias anyways). A highly voted on comment has >> 1 people who share the same sentiment and the beauty of sites like these is highly up-voted comments bubble to the top.

I was referencing my personal perspective.

I reserve the right to shut down my own opinions, especially the anecdotal ones. I have known my anecdotes to be 91.3% bullshit.

While we're ignoring the very specific statistical requirement to have at least a minimum of a significance level, could we please stop using the faux-grammatical prepositional-because?

Regarding upvotes as a proxy for "me too": an upvote doesn't mean "me too" in many cases, it means "this comment was something I thought contributed to the discussion in a meaningful way." Thus using upvotes as a proxy for a sample size counter, isn't even remotely accurate because upvotes do not necessarily correlate to me or anyone else sharing your sentiment. They only correlate to someone saying that your posting added value.

I upvote things all the time with which I disagree because the commenter made a strong argument and it added to the quality of the discussion. I downvote things with which I sometimes agree because perhaps the point was badly made, inflammatory or otherwise not enriching to the discussion. And then, I also downvote comments that use trendy I "haz cheeseburger" level diction. Of course, given my own guidelines, I probably ought to downvote myself.

"Yelp is all but meaningless in San Francisco, probably its most popular city. Really average or even crappy cafes, salons, restaurants have greater than or equal 4 star ratings... it's a little baffling. I presume from the stories you hear that it's because the businesses pay."

I have noticed this as well, but if you read the reviews you get a good idea of what is happening in SF: tourists.

Joe and Jane Honeymooners come to SF from Tulsa/KC/Duluth/DesMoines and have the "best meal/coffee/pastry/brunch EVER".

Good for them.

The flip side happens as well, especially with hotels on sites like Trip Advisor. "My $200 room in Manhattan was tiny and looks out on a light well." Umm, welcome to NYC!

Ouch. You consider a tiny hotel room to be ok because "that's how it is in NYC"?

If you're vacationing to NYC from Kansas for the first time, you might book the largest affordable room in the city and be shocked at how small it is. Conversely, if you are familiar with NYC and travel to it enough, you might recognize that a 400 sq. ft. room for $200 a night is an astonishing bargain.

Both types of review are valid, but the former is more valid for new-to-NYC vacationers while the latter would be better suited to those who are repeat travelers or people people who have migrated away from NYC and are vacationing home.

The complaint though, is that there's no way to distinguish between them. How big is big? How small is small? Reviewers seldom list the square footage, and the net result is that one person's anecdotal evidence is countered by another's opposite anecdotal experience. At the end of the day, unless there is an equal number of use cases, we're left reading reviews for context, which renders the star ratings useless.

I don't spend a lot of awake time in my hotel room when I'm in NYC. But my point was that if I have a cheap room in NYC (and $200 is quite a cheap room) my expectation is that it will be quite small.

I actually often stay at a hotel that has small [edit: 170 sq. ft] but deliberately engineered for the size rooms and I rather like it.

Funny thing, I tend to use Tripadvisor more for reviews than I do Yelp because I find the non-local perspective on places to actually be a benefit when it comes to deciding where to go. I am fully aware that those reviews are even more profoundly tourist-tinged than Yelp, however, there are some very smart travelers. For example, if some lady that has been all over Asia said "this was the best Kung Pao Chicken I've had since I was in Suzhou" then I'll pay more attention than some San Franciscan claiming their Kung Pao Chicken was the best one outside of Chinatown.

Also, the implication that everything is better in SF/NYC compared to Tulsa/KC/Duluth/DesMoines and thus the opinion of Joe and Jane Honeymooner isn't as useful is really rather snobby.

I find Yelp very useful in the following way: good places will exist and have lots of reviews on Yelp. Sort by number of reviews, and ignore the star rating.

How long before someone from Yelp reads this and adjusts the numbers to favour their advertising clientele?

I think it is a result of the five star rating system -- a vast majority of the businesses fall between a 4 and a 4.5 star rating. Perhaps a binary rating system is a better forcing mechanism to determine quality?

But since we're stuck with the five star system...here is how I leverage Yelp review data when it comes to selecting a small business to patronize:

1. Normalize for the true Yelp rating scale of 4 - 4.5 stars. Filter out anything below a 4 star rating. If a small business is unable achieve 4 stars on Yelp, then something must be really wrong.

2. Quantity of reviews is important. For starters, you need a significant sample size of review data for the star rating to be meaningful. Further on the review quantity front: massive review numbers relative to other similar businesses in that given city is a good signal of popularity (though not necessarily quality).

3. You need to supplement your Yelp findings from points 1 and 2 with other sources, ideally critical reviews (e.g. Eater, Serious Eats, Time Out, NY Times, Chicago Mag, etc).

4. The content of individual Yelp reviews should be taken with a grain of salt. Your individual priorities and tastes might be radically different so there is limited value in an individual's rating. I've come across some truly great restaurants with 1-star reviews because the reviewer tried to dine at the restaurant on a Monday...when it was closed.

This bothers the hell out of me. I reflexively rate anything on any platform 4 stars if I like it, leaving one star as "room for improvement" if I can think of any one thing I wish they'd done differently.

But the modern convention is 5 stars as "good", and getting too many 4s is a failure.

Normalize for the true Yelp rating scale of 4 - 4.5 stars. Filter out anything below a 4 star rating. If a small business is unable achieve 4 stars on Yelp, then something must be really wrong.

My experience is that higher star ratings on Yelp equate mainly to presentation over taste and that higher prices tend to mean a better rating (Out of a few average Chinese food places, the higher priced one will be rated the highest).

>higher prices tend to mean a better rating

If you assume an efficient market, they presumably wouldn't be able to charge a higher price if people didn't like them better. Of course, the average rater may care more about presentation than you do. A lot of people do go to restaurants more for the "experience" than for the food.

I wonder if a breakdown of services would be better. One to five stars for things like price, service, choices, quality, and so on. No overall one score for the entire experience with an often useless text description of what supposedly happened.

But, I think in the long run it doesn't matter. It just seems consumer based reviews are useless overall and people should stop relying on them.

Reviews and recommendations are harder than you'd expect.

I run the restaurants team at TripAdvisor and we're working to solve problems in finding the best place to eat. If you think you can do better, you may be right; come prove it!

Drop me a line, especially if you're excited to work on big piles of data, NLP, ML, a heap of phone stuff and crazy stuff like using word concreteness scores to analyze review value.

Oddly enough, in the Peninsula I think it is reversed.

You get restaurants that would be a solid 3-stars (maybe even 4!) anywhere else in the country, but because they are competing against all the hot new places that food snobs like me love, they are only 2-3 stars max.

It's interesting to compare to Japan's version, tabelog.com

IIRC the highest rated restaurant on the site is 4.62 (I might be off but it's not 5.0) Most restaurants are in the 3.0 to 3.6 range.

A curious difference in culture to Yelp

TripAdvisor's reviews have the same problem, but usually they've been faked all the way up to five stars. It's ridiculous.

And these days, it seems average restaurants are sporting the "TripAdvisor Certificate Of Excellence", so now that's meaningless too. I've seen it on one genuinely top-class Indian restaurant, but recently saw it on some distinctly mediocre places in Singapore.

The trick with TA is that you want to look at all 1, 2, and 3 star reviews and ignore the 4 and 5 star ones (just assume they're all fake). Then you know the "worst case scenario" with a specific establishment, the worst it can throw at you.

Stereotypically most negative reviews are about booking issues (e.g. lost booking, someone arrived too late and the booking was given away, etc). Or moaning about the free food or free/paid WiFi.

I don't think management replies add anything, unless they're being a jerk/dismissive. It is better just to say nothing than to be a jerk, since others reading the reply will assume that they will be treated the same way.

That sounds like a useful way of looking at the reviews. I'll try it, thanks!

TA does their own review vetting (of course), but it definitely seems like they're less selective than Yelp. As a result, they don't seem to get badmouthed by businesses as often, but that has downsides, as you point out.

In my experience, TA seems to have a larger self-reinforcing effect than Yelp: Top 10 places get more great reviews, stay top 10, even if they're dropping off in relative value or weren't that great to begin with.

Wrong, probably wrong and wrong. Yelp gets lots of usage from Sasn Franciscans in SF. I'd be curious to see a few of your 4+ star cafes that you don't think deserve it. And of course it has little or nothing to do with payoffs as all the dropped lawsuits demonstrate.

My Yelp story: I was a pretty active user. I enjoyed writing funny reviews of crappy places. One day I was trying to look something up and got a pop-over to confirm my account info. I clicked through and went on with my day. A few weeks later I received notice that my account was deleted. I contacted them and was informed I was too young to use the site and there was no chance of account recovery (I was a few years over 18 at the time). All of my reviews were gone. Emailed them, no chance of getting those back, though I was welcome to create a new account.

So, they managed to disappear all the reviews I left. They did it on a technicality and it was well within their terms of service. However, I view this as vote manipulation. They're pushing really hard now to say they don't do that. They burned me once though and now I actively avoid the service.

Yelp talks about not changing reviews or deleting bad reviews.

What obviously DOES happen is that they sort reviews differently and make the bad ones hard to find.

Then they come out and say 'we don't delete reviews, we don't manipulate them' but it's easy to find companies that have reviews sorted so that the negative ones are buried and on a completely separate page from the main reviews.

What you need is evidence to show that the way this affects companies differs based on whether the company is paying Yelp for advertising. There are so many reasons why either positive OR negative reviews may be filtered. I've seen false review vote-brigades on a business because someone from the business made offensive remarks about palestine and the military.

Used to love yelp but not letting me use the site on mobile browser is an absolute deal-killer. Apple should disapprove any app where the websites functionality is intentionally broken on iOS in order to force users into the app.

pinterest too. hate this

maybe it's just a way to reduce the amount of work that needs to be done. What the heck is wrong with downloading an app?

I don't have the space on my phone for every website that wants to be an app. The great thing about the WWW is that information can be accessed without having to install a special program for each provider. The web browser is plenty capable of handling Yelp's needs.

It might be to reduce the amount of work, but I still don't like it, and would rather they let me use my already-existing web browser to access their web site.

I think the legal definition of "manipulation" might differ from the "ethical" definition. What I think has happened here is that the court didn't find any "illegal manipulation." The question of "unethical manipulation" may still be up in the air. The problem for Yelp is that it seems to serve as a psychological outlet for a large number of people. They're angry about something, it builds up, and they then take immense satisfaction in channeling it into a business review.

Yelp doesn't manipulate reviews, their sales people tell you they'll remove bad reviews if you pay them money.

It's easy to behave this way when you're a monopoly.

What are they a monopoly in, millenial-targeted web reviews for specific industries?

"Monopoly" is not synonymous with "successful."

They're a monopoly in local reviews, a modern day Yellow Pages.

They had $377mm in 2014 revenue. Their biggest competitor is Angie's List who's now facing competition from Thumbtack.

Why isn't a monopoly synonymous with successful?

> their sales people tell you they'll remove bad reviews if you pay them money

Has this ever been substantiated? It seems like it would be a pretty easy thing to document.

I'm not sure. You'd have to record your calls before they call you.

Or you could just take a screenshot while you're on the phone, and then another one a week or two later. If the difference is as large as the reports make it seem this should be easy to detect.

Reply on news.ycombinator.com