Yelp, You Cost Me $2000 by Suppressing Genuine Reviews. Here’s How You Fix It
justinvincent.com
justinvincent.com
A company I used to freelance for found that after they stopped paying Yelp for a premium account, they started seeing more negative reviews and a number of their positive reviews were filtered out. They got almost daily phone calls from Yelp marketing types essentially suggesting that if they renewed their subscription, the negative reviews would 'disappear'.
This was a few years ago, so maybe things have changed since then. Either way, I wouldn't hold your breath for a better review verification system.
But if I don't trust the reviews, Yelp effectively degrades into a complicated remix of Google Maps.
Same thing happened again in Vegas, looking for a nice place to eat, find something that looks interesting and then see that Yelp places it in an industrial complex nearby. This time I don't expect to find the place, but I drive by and find myself somewhere in a dark alley full of closed-for-the-evening auto shops.
I'll still use Yelp to find the name of a business, but then I have to switch to Google Maps and do a reality check to actually see if the business is near by and how to get there. A real PITA.
EXTREMELY worth mentioning. If you've yelped Yelp, so to speak, you know to take their recommendations with a grain of salt and then some. Caveat emptor.
While I'm on the topic, the idea of a single company managing reputation for any given space is silly beyond words. Downright moronic, even.
Reputation is an aggregation of multiple opinions. No aggregating entity can aggregate without their aggregating principle spinning the results. Yelp is notorious for a corrupt and indeed outright criminal aggregating principle -- "we'll suppress bad reviews if you pay us and good reviews if you don't" -- but the same is true for any aggregator, whether they're aggregating based on good principles or bad, because aggregation is a form of authorship. I mean look at Hacker News vs Reddit vs Digg vs Slashdot. They all aggregate, and each has a distinctive voice.
Because of this, yes, Yelp is a useful resource, but no, don't use it for a serious purchase without going on Google. If you use Google and Yelp, you're at least polling multiple sources, so you're doing something which can access the fundamental aggregating dynamic which powers reputation in the first place. A single point of failure for reputation is a contradiction in terms!
A review aggregator website where the value of reviews from various sources is weighted by how reliable they are. For example, if there is Yelp, Facebook and Google, it can weight a restaurant 0.7 Yelp, 0.2 Facebook and 0.1 Google.
we do. if you use Google as well as Yelp, every other site on the Internet which mentions the business you are curious about effectively acts as a separate review site.
http://abcnews.go.com/Blotter/business-bureau-best-ratings-m...
Each court case related to this has been thrown out.
If you trust Google to be fair with PageRank then there hasn't been any evidence that should make you more concerned about Yelp.
Business owners complain about getting demoted in Google's PageRank algorithm quite a bit, too.
> Business owners complain about getting demoted in Google's PageRank algorithm quite a bit, too.
Google rankings fluctuate over time, but it is never the result of a direct targeting of an individual company or URL by google, whereas there is ample evidence that Yelp directly targets individual businesses depending on whether or not they are paying customers.
Can you link to any of this "ample evidence"? Preferably not just allegations from business owners who don't like their reviews.
Yelp's filtered reviews fluctuate as well. Reviews that are filtered can be unfiltered if the system decides the reviews aren't actually spam.
http://chicago.cbslocal.com/2012/02/27/2-investigators-busin...
Wait, Yelp gets money from the businesses they list? Yeah, that's not a conflict of interest.
The actual reviews are here: http://www.yelp.com/biz/royal-transportation-moving-and-stor.... It's pretty obvious there is something fishy about the unfiltered reviews. Each reviewer first left a one line review "Amazing!!! one of a kind..one of the best sushi place" and "the best of the best" before their very detailed, glowing review of the movers. If that's what it takes to fool the spam filter then something is very wrong. By contrast one of the filtered reviewers has ten measured, thorough reviews but his review has been banned for ToS violations.
Yelp really needs to disclose which vendors have a business relationship with them. They should also make the filtered reviews easier to see (place the link next to the unfiltered review count) and remove the damn CAPTCHA. I regularly try to check the filtered reviews but can't because the CAPTCHA is too difficult to read.
really really really makes me suspicious. Just did a check to make sure that google does not integrate yelp reviews (it does not) since that is what I mostly use.
also worth noting that "bob" liked the company so much he cross-posted his review to google.
Rather interesting to see his google review history, as long as we are speculating . . .
http://maps.google.com/maps/user?uid=213714492309658467573&#...
_____
edit: also a bit of fun to compare these two reviews:
http://www.mymovingreviews.com/movingreviews/myself-and-my-f...
http://www.yelp.com/biz/royal-transportation-moving-and-stor...
They found that there were two patterns of content contribution: people who wrote one review and people who wrote lots. For people who wrote one review, anonymous contributions were higher quality than signed, and those single reviews were higher quality than people who contributed multiple reviews.
Chances are good your most valuable reviews are coming from the long tail and your least accurate reviews are coming from people who are posting lots, but potentially with sockpuppet accounts. Tossing out your most accurate reviews in an attempt to suppress the latter seems counterproductive.
In a nutshell the problem is that people who are not verifiable by "the service" provide information or data which is essential to the operation of the "the service." Further the beneficiaries of the service (customers) are not paying for the service, and the people who can be harmed or benefit from the service (establishments) have every motivation to game the system to increase business and little downside.
So your business which has modestly thin margins to begin with, grows slowly, and the more popular it gets the more it tries to get gamed and the more margins are squeezed if you're paying people to filter those reviews.
There aren't really any 'good' answers. But there are other examples of businesses like this that have done well. The ones that do well however use the 'review' portal to push more customers to the 'value' area where higher rates of returns are to be had.
I hope someone writes up Yelp, Ripoffreport, TripAdvisor, Expedia, and AngiesList up for a Harvard Business Review group study. There have been a lot of experiments in this space taking different tacks and it would be interesting to compare their effectiveness.
They just called us up a couple weeks ago and sat on the phone for over a half hour (because we were willing) taking reviews from us on a bunch of companies that we had used.
This is why it costs money; they actually provide a service. If they don't have enough reviews, they hire people to call you up and write up the reviews for you.
The Limbaugh thing is unfortunate, that's true.
But they provide a good service, and they also support NPR. I am under no illusion that every company I do business with will agree with me politically in all ways.
It's not entirely fair to criticize Yelp for under-emphasizing negative reviews.
One of the big hurdles to overcome on a user ratings site is that people are most motivated to write reviews when they are outraged. There is a massive selection bias, and if nothing is done to alleviate this, your site runs the risk of becoming a soap box for ranting.
You work harder to get positive reviews than negative ones. If you receive 2 bad reviews and 2 good ones, that does not mean 50% of people are dissatisfied.
This should be the motivation for preferencing good reviews over lots of one-off bad reviews.
BUT: Yelp made the cardinal sin of "objective" ratings sites: their profit comes from the entities they are rating. Ergo everything on their site is questionable.
Like you say though, Yelp is just making the playing field uneven in favor of those who pay to play, so none of this matters anyway.
There's a great big gap between letting all your angry customers (which hopefully you don't have a lot of) leave rants and creating sockpuppets so you have nothing but great reviews. The space between is the basics of word-of-mouth marketing: ask your happiest customers to talk about you.
No one is stopping American Eagle Movers from asking their happy customers to head on over to Yelp and leave them a review. They just don't seem to have any happy customers. So they have to fake it. And then Yelp steps in with their [seemingly] shady selection practices. Hence Justin's disastrous experience.
The only one that will leave a good review will be Yelp users that rates everything.
And maybe people that you did them a big favor and they feel they have to repay you someway.
Good reviews are worthless to me as a consumer. What I want are details, good and/or bad, that I can use to evaluate my purchase ahead of time. "This is great!!!", "I would use it again!" are devoid of information.
If negative reviews are more informative, then I assure you, you should welcome them over a bunch of five-star "great!"s.
Besides being informative for me, the user, they should be valuable to you, the business owner. Nobody maintains an "attaboy" database, but you better keep track of bugs and fix them if you expect to grow or even survive in this economy.
Legitimate complaints are welcome, I never implied they were not.
The thing that Yelp that makes it stand above other review sites is that they incentive positive reviews. Like Yelp or not, the idea of "Yelp Elite" is actually a great idea, because it is able to draw out a lot of medium to great reviews, stuff you would rarely see on other sites. When you incentive reviews, either good or bad, then you do tend to eliminate this "one-and-done" bias that tends to focus on the negative.
That being said, my wife is/was a Yelp Elite for several years with over 800 reviews, but quit this year because she thinks that Yelp as a whole has started to go downhill, and doesn't like the pretentiousness of most people on there now.
What's tricky about it now? Click "X filtered" reviews and you can read to your heart's content.
I'd imagine Yelp would rather not have a "Yelp Unfiltered" extension made and their filtering becomes less important. Same for helping spammers automate their processes by checking up on their reviews.
It's a race to the bottom though, the spammers will eventually win.
This is the risk we take listening to ANYTHING depending on voluntary responses. Burying negative reviews to attempt some intrinsic rating dismisses the value in actual user input, with the paternalistic view that the review aggregator somehow knows better. Who decides which and how many negative reviews to hide? You? How about letting the user figure it out, knowing that trusting any opinion is a risk and learning from experience to calibrate reviews he reads to himself? It's the best we can hope for without resorting to the review site, in effect, clobbering consensus views to get something it "knows to be more accurate." In that case the users no longer review, but the site, which is not a user, does.
> If you receive 2 bad reviews and 2 good ones, that does not mean 50% of people are dissatisfied.
You absolutely do not know this. It could be that 70% were dissatisfied. You can't assume this without taking the role of a reviewer, which you cannot legitimately do without experiencing the item being reviewed.
Q: How can you verify the quality of a < mover / restaurant / hairdresser / babysitter / dentist / chiropractor / handyman / etc.>?
A: Ask your < friend / relative / neighbor / workmate / schoolmate / etc.>.
2009
Q: Same question.
A: Don't need no stinkin' people. We have the internets!
2012
Q: Internet solution doesn't work. How do we fix it?
A: Use the 1995 solution.
If a friend recommends Avengers Assemble, I go see it. If I end up at the cinema out of boredem, I'll let imdb help me.
I want some website to come along which solves this for me, and neither Facebook nor Hunch nor IMDB do really.
We're professional curators. That means every restaurant review (and shops, hotels, etc) has been written by an actual human professional who's opinion we hope you trust (we're a little more upscale than Lonely Planet, but we have our hip moments). So in a way, we're the anti-UGC. When you have crowds of people, all the ratings become a noisey 3-4 stars. Including Burger King. That's why I think Fodor's content is a good alternative. We're in the travel space, not local, but, it's a refreshing alternative.
Oh, and all our apps are FREE.
The other problem is that not everyone is the same. For example I absolutely hate "attentive" staff in a restaurant (let me eat in peace and if I want something I'll ask for it), but want the check as quickly as possible on finishing while others like to slowly wind down. A regular reviewer probably won't mention either of these because that is normal to them.
The existence of multiple reviewers means one will be more like me and would mention these issues which would make me go somewhere else while others who enjoy that kind of experience would be more likely to come. This doesn't make anyone right or wrong, but does show how different reviewers help and why the lowest review scores have the most informative reviews.
Reviewers may prefer attentive staff, but the good reviewers (which exist) know that not everyone shares their preferences. That's the difference between a good review and a bad review: you can translate a good review into your own viewpoint and decide for yourself.
When I think of noisy one star reviews, I mostly think of the one star review for a microprocessor where the guy didn't use a heatsink and the motherboard melted. There are plenty of folk who come across well in writing but are monstrous in person, I'd never expect to trust their restaurant reviews, and I might not be able to identify them either.
I love reading Ebert's movie reviews. I often disagree with him, but he is a mostly fixed point. I can tell if I'm going to like something because, over time, I have developed an understanding of where his tastes and mine align and where they differ. This kind of thing is a lot harder to do online—it requires real work and attention.
In a way, we're disagreeing about where we want to have our statistical anomalies. Do we want the reliable reviewer caught on a bad day, or perhaps recognized by the staff? Or do we want the sea of one-time reviewers, a few of whom had a statistically unlikely bad or amazing experience?
Man, I'm 30 and this makes me feel old. :) In the print travel guides market, it's one of the big names. It comes from the generation before the more backpacker-oriented guides (Lonely Planet, Rough Guides, etc.) and targets more of an upper-middle-class family vacation type of audience, but it's still fairly popular, within the print-travel-guides space.
It does seem that their move into digital guides is quite recent and incomplete, though.
Not for Nokia, they aren't. (AUD 6.60 each)
Q: Internet solution doesn't work. How do we fix it? A: Use the 1995 solution but because it's not 1995 anymore you'll use Facebook and Twitter to ask your friends which is still using "the Internet".
But that kind of blows up your point/argument.
But not in the "back to square one" meaning, it is rather "next level of a spiral staircase". I believe there's still room for improvement (definitely there is over the '95 low tech and probably over what we have now) in the way we can combine internet and the '95 solution to find good <mover / restaurant / hairdresser ... etc>
Time will tell - lots of money goes into social, someone will do it right or at least we'll know what obstacles prevent this from happening.
If in this case the problem were likely to be solved by acquaintances, it would also be likely to be solved by Yelp, under its assumption that people need to leave multiple reviews to be considered credible.
We're so out of touch that people are talking about how this space is ripe for disruption and a new startup. Umm, Yelp is an extremely young company. I guess once you're public you're seen as old hat? Pump and dump, and move on to the next thing. Unbelievable.
This is why I've gone off sites like Yelp for recommendations, and instead stick to places like Foursquare which rely much more heavily on the behaviour of the people that I know. Seeing that several of my friends have been somewhere more than once is a much stronger recommendation signal than a bunch of strangers giving something five stars.
It will be quite easy to tell which reviews are real and which are not. At the very least it will make the company look questionable because 50% of reviews will be 5 star and 50% will be one star. The main issue is we need to get valid reviews out of the filter so that Yelp users can more easily make their mind up.
As it stands a lot of real reviews are hidden and a lot of fake reviews are exposed. Simply by exposing real reviews you level the playing field.
Besides, I never supplied sufficient information for Yelp to connect me to my credit report without doing some seriously ethically questionable data mining.
Why do you believe that Yelp does this? If so, that would be more surprising to most users than the pay-for-play behavior we're seeing here.
That said, I think some of the problems with the credit report solution still stand; namely that pulling credit histories has a cost (both to the reviewer and to Yelp). I would worry about what they would do with that information beyond merely authenticating me as a "real person" and I would expect that fewer people would be willing to review a local donut shop if doing so required a credit pull.
Every business can have bad days, and a negative review can put a prospective customer off even if actually 99% of the time that's a good business.
This isn't really a comparative recommendation site like yelp, the scenario would be something like a user saying what they want to do (like eat burgers on friday) then people would only recommend place to go. Kind of like Quora for places.
Eventually you could build a decent picture of business that are recommended multiple times to start making suggestions beyond the crowd sourced suggestions.
Would you mind looking at (the web site listed in my profile) and describing what is still missing?
edit: removed link
FWIW I watched the video (all the women have moustaches!) but am no more informed as to how it works. I also looked at the business sign up page but couldn't find any thing there to help work out the mechanics of the process. It looks like you integrate with FB closely?
I was unsure of the etiquette here at HN about pimping one's own site in the body of a post. I saw that others chose to put their site URLs in their profiles, but very few were posting them in comment text.
As 'iamgilesbowkett mentions above, maybe it would be more informative to search over an aggregation of different opinions rather than hash mountains of conflicting reviews into a single byte digest.
For the consumer, our site offers a way to casually discover nearby businesses based on how well they are liked by specific circles of people. All activity is contextualized by location; the target of attention is geo-tagged by lat-long, not the user.
We try to reduce the effort required to browse through large amounts of business listing and business review data. The user should be able to quickly explore a specific area for businesses that match a search term, or are physically close to another (perhaps unrelated) business.
Please feel free to surf around using the business search.
On the site, business search itself is open to the world, but the core social parts require an account to participate. You can enter an object term plus a location. Search URLs can look like this: http://spotlikes.com/discover/places/noisebridge/san-francis...
Due to the founders' interest in Napa and Sonoma counties, we have lots of comments describing San Francisco North Bay Area small businesses. One entry point for browsing might be someplace like: http://spotlikes.com/business/benziger-family-winery-glen-el...
During after-release QA, sometimes I try to do a kind of major metropolitan area hopscotch, visiting all the largest US cities using the fewest clicks.
For the SMB owner, our site offers businesses a place to make a simple listing - a page where they can post their basic details (phone number, URL, logo, etc) - and then sit back and watch the users stream that small referral link out in their activities.
Yes, the site is tightly integrated with Facebook. User comments on business listings can flow automatically into social media sites. If you authorize it, your "likes" will flow onto your FB wall/timeline. We have similar Twitter and Foursquare integration.
We have probably overlooked all kinds of stuff. We move pretty fast. If you would like to call out anything in particular, I am keenly interested in the feedback.
Of course, if I want to look a specific place or product, pos and neg reviews are useful.
What people review on Yelp is subjective and Yelp makes it hard to filter out people's biases because of their one dimensional rating system. When people review a restaurant, they review the service, the food, the atmosphere and the overall value. For some people, paying $200 on a meal is outrageous regardless of how epic the food might be. For others, they don't even look at the check when they pay and value doesn't factor into their ratings.
Yelp doesn't ask you to rank the businesses you like on a scale of best to worst, so people rate things based on some internal weighting scale that probably shifts slightly from review to review. I may have a 5-star taco trunk right next to a 5-star review for the French Laundry. They aren't equal.
What people review is also temporal. God help a business if there is say, some construction going on across the street, the patron was in a bad mood and becomes argumentative or the chef is sick and food quality dips for a night. Or maybe the waiter just broke up with his girlfriend. Maybe they run out of the one thing the patron really wants to eat. Any of these things can make someone walk away and feel the business is poor.
Businesses and people also change over time, I have reviews on Yelp for 5 stars from 5 years ago for places that have either gone downhill or simply don't match my current taste.
People also tend to only review things they either really liked/loved or hated which results in most businesses (especially in SF) as being rated 3.5-4 stars making Yelp pretty useless without looking at the underlying reviews.
A collaborative filtering engine could fix a lot of these problems. At the very least I'll match people who have my own biases and normalize people's ratings. Of course it still doesn't fix the temporal issues, but I don't think that's insurmountable.
Do you think that you would see better results if Yelp was able to use a collaborative filtering approach to somehow personalize their results such that they more closely match your tastes and preferences - based on other users' data?
Example:
"We were really worried about where to stay on Las Vegas, but then we called hotel X and Margo sounded so nice and welcoming on the phone that we just knew it was the right choice."
For you, but not the vendor. See PayPal.
Question based identity verification processes are already used and supplied under the hood by agencies such as TransUnion, Equifax & Experian.
And that leads to friction. Yelp got it's massive user base from frictionless technology.
I don't like Yelp's business practices just as much as the next person, but this is a human interaction issue that isn't solved by a simple algorithm, nor is it solved purely by (mind you, costly) human being based committees.
Assume people with a 4-5 star review are competent. GET MULTIPLE QUOTES. Doesn't matter if they have 5*, if you feel something is off with them they you avoid.
You also will find out when a quote is too cheap/too expensive and reduce your risk to getting conned.
Don't blame the internets or Yelp. Blame youself for not doing due diligence.
Is anyone surprised that a recommendation site that gets its money from the businesses is less than honest? Is anyone surprised that when they get popular they turn into a protection racket?
Say it with me. "If you're not paying, you're the product." Or in this case, the club.
If you guys made a competitor to Yelp, how would you improve on it?
I'm not sure how it compares to other sites, but you can see the URL in my profile.
The hard part is staying one step ahead of less-legitimate accounts. There's always somebody trying to cheat the system, and stopping these users often results in a tradeoff.
But a continuous, consistent pattern of one-shot negative reviews is also not particularly difficult for a competing business (or anyone with a grudge) to reproduce.
This is a hard problem, and I don't think the credit check-style identity verification questions suggested in the post are going to solve it. For one thing, I think an overwhelming number of people would find it creepy. They're trying to tell people about a bad moving company, not get a car loan.
Putting more of your "valuable" online identity at stake seems likely to help. I hate to say it, but "Log in using your Facebook account" suddenly gives Yelp a lot more data to exploit to decide just how much trust they should put in your reviews.
As far as the user is concerned it's just a few questions and answers.
If the questions were about my credit history then this is a deal breaker.
There is no way a site like yelp can get me to authenticate by answering questions about my credit history and I suspect that the vast majority of people would agree.
Now, facebook credentials, that's a different story.
Just a few questions and answers that could hint at all kinds of things including relationship status, orientation, race, financial status, etc. Imagine if some stranger on the street, or even the cashier at a trendy retail store, asks you about some detail they have no business knowing. You would find it creepy. Now imagine it's some random web site (which is all Yelp is to most people).
Putting in my credit card (or revealing any information I don't deem relevant to the company / form) is always an immediate deal breaker. For Yelp, a critical mass of reviews is needed, I'm not sure they can afford that.
I have a friend who is currently a target of such a vendetta. He got into an edit dispute with someone who was trying to censor stuff on a wiki. This other person responded by flooding listings of my friend's previous employer (which has been out of business for a few months) on nearly every review site out there with a continuous stream of utterly fabricated reviews. These reviews range from calling him a pedophile (easy enough to get deleted) to pretending to be a client who was screwed by his incompetence. I'm not sure there is any way to differentiate the latter kind of reviews from genuine negative experiences.
The main problem here is that Yelp is a racket as well, this has been known for some time. Those using it, well caveat emptor.
There are other review sites out there that are not as morally compromised as Yelp due to their business model being modeled on a mafia protection racket of "Pay us and we can help these bad reviews go away."
That defeats the purpose of an online review site -- if the review is only visible to me, why am I posting to Yelp, and giving them valuable data they just will sit on, as opposed to logging in my own local host `reviews.txt`?
I noticed also that a number of favorable reviews, seemingly written by authentic reviewers were also suppressed in like fashion.
See Amazon for a site that gets posting reviews mostly correct.
Featuring the 1-stars and burying the 5-stars would also be quite the incentive for a business to pay Yelp, but it would be cynical to assume they are doing that, too.
Five stars review system is flawed.
I find that to be the most shocking statement in the article.
If you're not a customer, you're the product.
Example: consumer goes to idverificationsite.com, registers there for free- they ask out of wallet questions to verify identity, potentially do a minimal background check too.
You then use this login with Yelp, AirBnB, Getaround, etc.
Yelp, Getaround, Airbnb pay idverificationsite.com some small amount per verified person. Removes some of the friction of ID verification/spreads cost around.
Sidenote: the BBB is the worst offender of these protection rackets - http://feefighters.com/blog/bbb-accreditation/
Also - I've just decided to start posting reviews to Google rather than Yelp, they lost me as a product after reading this thread.
It would be nice if this service was available more globally.
http://www.inc.com/magazine/201112/case-study-josh-levy-and-...
Several years ago, I was working with Honesty Online, doing exactly this identity verification startup prop and where it became very evident that this is really two businesses. Now we, in another startup, sigkat.com, are very early stage in the second business - creation/exchange/transmission of any sort of credentials so that vertical apps like identity verification can focus on their domain.
So how could that work?
This article just proves how stupid consumers can be when making important decisions.
When I read the article I didn't see the guy mentioning looking up the company on the BBB or even simply Google'ing "[Company] scams" or "[Company] scammers".
Yes, Yelp could implement better solutions to protect consumers but ultimately the consumer needs to do adequate research before purchasing.