Still trust restaurant reviews? | Hacker News Reader

Still trust restaurant reviews?

blog.urbantag.com

47 pointsakitchell45 comments

Some thoughts on why a new review ecosystem is needed, and about to develop

I trust Yelp/Urbanspoon reviews the same way I trust Amazon reviews: By reading them and looking for clues that the reviewer is real and has astes and concerns similar to mine. I gloss over uncritical reviews, whether they're real or not; a review that says "everything was great but the fries" I'll put stock in.

It's possible that astroturfers could start taking that into account, but it's hard to imagine. The kind of people who stuff Yelp reviews generally have personality quirks that would prevent them from allowing any negative feedback.

Yeah, Viewpoints Network tried the approach of "reviewers similar to me" and nobody really cared. Now they're just another opinion engine.

I would guess this is what a lot of people do and it's certainly what keeps me from giving up entirely on Yelp/Urbanspoon. But the downside is that ranking results by average review becomes useless; you have to dig pretty deep to find the place with an average of 1.5 stars that has a couple of thoughtful reviews indicating it's somewhere you might like. (i.e. you have to get your reviews in linear rather than constant time :)

Companies like Hunch are addressing the problem from the social-networking standpoint that the author suggests (their Local app adds a Facebook friend layer on top of scraped Yelp reviews), but it seems most people are not yet dissatisfied enough with Yelp to consider getting themselves invested in yet another social networking service.

Like you I also check the quality of the reviews. However, I do cull my list initially based on summarized data.

What this ultimately means is that when looking at a map or listing with sites and summarized data you can't trust the summarized results which may have been gamed/poisoned.

Also poisoning the summary is very powerful in controlling which users even notice the property (e.g. 25 reviews 3.2 stars - might be worth looking at, 15 reviews 2 star = probably avoid... the difference is 10 bogus 5 star reviews, not hard)

black-hat SEO works

I ended up with an excellent recommendation for a Chinese restaurant in Milwaukee, by reading past the lame Milwaukee Journal survey article to the comments. One commenter provided enough context, detail, and personal investment -- in the topic, but not the restaurant -- that I felt confident taking the recommendation. And he was right.

This still seems to be the only way to get a good recommendation out of the web. It puts us in touch, but we still have to pay attention and filter and aggregate manually.

Part of the point of the review model of Yelp (and Amazon, for that matter) is that you don't have to trust a review. You only have to trust the consensus of all of the reviewers - which, at best, is very, very hard to spoof. When I see a place on yelp with a 4-star average on 1000 reviews, I feel very confident about going there. It's hard to get to that critical mass, but far from impossible - and you don't need someone's "real name." This is actually somewhere where Yelp is doing well, too - they give people a _persistent_ identity. You should trust reviews from someone who's written 100 reviews more than a review from someone who signed up just to review a single place - and Yelp makes that judgment easy by putting the information right in front of you. Blizzard and other strategies have demonstrated that real names aren't necessary for a good community - just persistent names.

Also, I have to laugh that the UrbanTag folks approvingly cite the complaint that existing reviews sites are "giving customers “the upper hand” in determining an establishment’s reputation." Well, yes, they are. That's the point. That power has always rested in customers' hands, because that's what reputation is: what other people think of you. You can work on your reputation and "manage" it and "optimize" it, but if it's not what other people think, it's not reputation. Business owners neither need nor deserve control over other people's opinions - look at those asshole doctors that were linked here a while back, forcing customers to sign agreements that amounted to "we can sue you for posting a negative review of us on Yelp."

As for unreasonable customers: that's nearly as overblown as the threat of piracy, and if your business can't deal with unreasonable customers gracefully (not the same as caving and giving them what they ask for), you deserve to go out of business.

Dealing with ungraceful customers is #1 thing I look for in a business. I am not usually ungraceful. I always try to be polite and friendly with any customer service rep, etc. However, I think it says a lot about a company.

For example, a budget hosting provider that I used for a while would go off on any customer who posted a negative review of them in the comments section of a blog I frequent (LowEndBox). They turned out to be a terrible provider and even being a reasonable customer, I could not get anything from their customer support.

This is the same principle that can be used at job interviews. Take the interviewee out to lunch and see how they treat the wait staff. Quick way to separate assholes from people that will gel well with your team.

I know this is how most people feel, and I do try to pretend to be a sane, rational person who doesn't get angry when dealing with customers, and I think that helps me a lot. Flying off the handle at a customer almost never makes business sense.

But I don't think it's such a good overall heuristic for picking service providers. Personally, I'd rather deal with an employee who was good technically and bad socially than deal with someone who is the other way around. (I mean, if I could get both at a price I could afford, that'd be perfect. but that's not usually how my world works.)

If the food is good and the service is fast, sure, it'd be nice if the waiter was also friendly, but if I had to sacrifice something; price, speed of service, quality of food, or politeness of service, I'm going to pick the rude fast waiter and the good cook every time.

In large numbers, yes, it's hard to spoof a good/bad rating. But the mode/median of review numbers is likely to be easily game-able (like 20 reviews, for example).

Furthermore, if your property gets too many bad reviews you can either rebrand (expensive) or just create a new property description (this is what happened when my sister found a screw in her food - the restaurant retaliated to her Yelp negative review by just creating a cloned restaurant listing then spamming that listing with positive reviews).

That tactic, "reputation bankruptcy," is far more effective for individual persons than for a business, and in this specific case, it's pretty easy to programmatically detect. So I don't think much of that scenario.

However, yes, it's easier to game the numbers when the volume of reviews is low. That's an ongoing problem - for everyone.

Review sites are hard.

Yelp was wonderful when it first started because the people on it all loved restaurants enough to actually want to sign up to some random site and review them. The community was small and I was easily able to pick out people I trusted and people I didn't.

As it grew, more of the general public started reviewing things and things started to go downhill. No longer can I remember who has similar taste as I do or who to trust. There are so many reviews that it has become rather useless. Every business seems to be 3-4 stars from Michelin two star restaurants to the taco cart on the corner.

Worse, ratings change over time. Restaurants I rated as 5 stars years ago no longer qualify as 5 stars today, so even when I see someone I know, I have to look at the date of their review and discount appropriately.

Compounding things, there are all sorts of reasons why people give negative ratings from food delivery being late (which has nothing to do with your experience going to a restaurant) to having bad service because the restaurant opened the day before to not feeling like you got enough subjective value out of it.

Now, if Yelp had a collaborative filtering engine ala Netflix or only showed reviews from people who shared similar taste as I do, it might regain some of its usefulness. Maybe if they did something ala Hacker News and required constant up votes to keep the rating from falling over time? Or an OKCupid-style personality matching matrix to tell if someone is just cheaper than I am or demands a higher level of service.

I don't bother with 5 star or 1 star reviews. Too much emotion to be sensibly reviewed.

Three stars provide a nice balance. Things like "food was good, but service was slow." I find more valuable than a 2 paragraph review about how a waiter spilled a glass of water and only comped one entrée or a five paragraph review that can be summarized as "I love this place more than breathing."

Agreed - and we think your friends would benefit from those short, honest, insightful comments as well.

Its all about providing an environment where people you care about can easily find your recommendations... so that when you do find a 5 star place, your reviews will be taken seriously.

I like to read 1-star reviews (for anything) just to get a rough idea for what a 'worst case scenario' might look like for that particular thing. If the 1-star reviews are all about entitled patrons, that's pretty encouraging, whereas a few reviews about rats or cockroaches would make me look elsewhere. To use another example, I was looking at books on Amazon yesterday, and the only one-star review for a particular jQuery book was from this guy who was mad because the book didn't explain how to put quotation marks inside another pair of quotation marks (it was NOT a Javascript book) or how to download Firebug. Whereas another book's 1-star reviews might be about how the examples don't work, core material was skipped over, etc. When I was looking for a travel mug I could throw in my purse, I nixed any option that had a 1-star review about leaks (including many which were advertised as leakproof), and ended up getting the best travel mug I've ever had. I can shake it upside down or toss it across the room and my coffee stays safely contained. I think the bad reviews of it were about how hard it was to clean and how the lid came apart too easily. (It's supposed to come apart, to make it easier to clean...)

I do the same thing and I'm pretty sure I have the same mug.

Which mug? Tossing a full cup of coffee across a room sounds fun.

For some reason I can't find it on Amazon right now, but it's this one:

http://www.thermos.com/product_details.aspx?ProdID=282&C...

> To solve the anonymous review problem, its time to inject real identities into recommendations.

People are usually more honest when speaking anonymously -- not the other way around. It may help with astroturfing and flaming temporarily, but competitors and trolls will always find a way around real identity online.

Interesting point - what if your reviews are shared between your friends? Our findings are that people can be honest in a community where they know their recommendations will mostly help their friends find great places.

i sometimes don't trust my friends' recommendations. what if my friends and I don't share the same tastes? or, maybe my friend recommended a sushi place, but it was the first time she tried sushi...

The problem with that is I don't need a digital way to manage word-of-mouth from my friends; I can handle that mentally. I need a way to find out about places my friends haven't been to.

Interesting & strong point.

We're looking to solve that problem as well -- its a darn big challenge!

Take the number of your friends. Subtract the "lurker" ratio, which most studies show to be pretty high. Now subtract the ones who have never been to the place you want to see the review of. Now subtract the ones who went there but didn't review it (because do you leave reviews of every single business you use? if so, where do you find the time to do anything else?). The mode of the result will be 0.

If your friends are homogeneous and outgoing, you might achieve the critical mass necessary to have more than just one review on a handful of local businesses.

On Yelp, I have many local places with 0 reviews period, let alone by people I know.

Broader comment: Tying identities to reviews won't do squat. If I tell you I'm Bob Thurmond, put up a photo of an older white guy, and exercise even a modicum of identity management, who's going to be able to tell otherwise?

That, and often I need recommendations when I'm not in my usual place, like when traveling. The odds that my friends would 1) use the service and 2) review a place I'm interested in are very low.

If I want a recommendation from my coffee aficionado friend... I'll just ask him.

Persistence is what is needed, not anonymity. I don't care if the guy that wrote the review is of the opposite political persuasion than me. I just care that the last 3 restaurants where I ate on his recommendation were good, so the 4th is probably good too.

EDIT: I am suggesting using sha1(full_name + salt) as the displayed name.

>People are usually more honest when speaking anonymously [...] It [real ID] may help with astroturfing [...]

In the EU astroturfing is unlawful/illegal (not sure which) under the Unfair Commercial Practices Directive (UCPD).

I acknowledge your point about people having the courage to be more honest when speaking anonymously, however I suspect that mostly applies to public fora. If you consider limiting scope to only people you know or trust, I find comments can be much more valuable and transparent. Furthermore, humans are really good at applying personality filters to people they are familiar with... (disclosure: I work at urbantag)

Why doesn't collaborative filtering work for popular review sites? The Netflix recommender is pretty solid. Obviously you aren't likely to have reviewed 200 restaurants in Yelp, but it still seems like it would be effective if you had 10-30.

But if I could go to Yelp and have it recommend restaurants based on my past ratings (well I have none now), that would be nice. I'd even take it a step further. I want recommendations of actual dishes.

I always feel dirty when I read a topical blog post and find out that its actually just marketing material for the author's product.

I'm solving the same problem at http://tattle.com.

Our pitch is "Local business reviews and recommendations from people you trust."

For me, as a vegetarian (http://tattle.com/#communities/vegetarian), the problem with Yelp has always been that I don't have any good way of finding vegetarian-friendly restaurants. They are good for factual queries, but trust is a delicate issue. That's mainly because when you run a search for "vegetarian" on Yelp, you're getting back a list of places that have the word in their name or description, or occasionally a review. But actually, places are not vegetarian, people are vegetarian. So if you want to know what place to go to, you need to know who goes there. That's how we're solving the trust gap (in addition to other problems with reviews).

Have you looked at http://vegguide.org ?

Just did, there's also happycow. None of these sites have very much content, they clearly are very old, and tattle is not tailored specifically for vegetarians and vegans - we're just good for those types of communities.

The signal-to-noise has worsened appreciably on Yelp to the point where I rarely look beyond the star rating and the number of reviews, and even then I take the star rating with a grain of salt. The folksy, storytelling style they seem to have championed makes reading through even a sampling of the top reviews unbearable. It's too bad - I think they could successfully "pivot" (sorry) into everything from location to coupons relatively easily if their community didn't suck.

That said, Yelp is already fairly well integrated with the social graph (via Connect) and one of the best use-cases for Yelp is for finding destinations away from your main locale and, presumably, your social graph. I don't think this approach will add value.

I agree with most that reviews are generally prone to abuse. (Although frankly some of the Amazon non-serious reviews are just priceless, like the Cat in the Hat ones [1])

I am also surprised Facebook hasn't jumped in here, seems a really useful product would be 'facebook reviews' which linked reviews to profiles. Then 'the set of reviews you trust' could fall right out of your friends list.

[1] http://www.amazon.com/Cat-Hat-Dr-Seuss/product-reviews/03948...

As someone who had dabbled in this "social recommendations" space, finding reviews from friends when you do not have a big enough social graph is probably the biggest difficulty. Also, it may work in a place like SF, but outside of those deeply networked places, it becomes difficult to find restaurants with reviews on Yelp, forget finding places where your friends have been to.

And remember that people are far more inclined to write reviews when they have had a negative experience, and not a positive one.

A tough task. I wish you all the best.

Thanks Cherian

Would love to talk further about your experiences.

Feel free to write me - ak@urbantag.com

Thanks! and yep - def a tough problem... ;)

I also find Yelp pretty useless at this point, so I've been hoping to find a site that allows me to easily see the places that my friends (or friends of friends) have been and to see their reviews of those places. Are all reviews on your site going to be public, or is there going to be an option to only show them to your friends and their friends? I'm hoping it's the latter.

Thanks for your feedback. I think you'll like what we are up to! (I work at urbantag) If you have an iPhone, contact us and we'll prioritize you in the beta (reference this HN comment).

Trust? Nobody has ever 'trusted' reviews by other people. They've used them to help make decisions, but 'trust'?

I use reviews to find the negatives of something, and then decide if that negative bothers me. Filtering out people with an axe to grind can be an art all in itself, but many people are pretty easy to spot.

Yeah, there are two types of reviewers:

1) I review every place I go.

2) I want to punish/warn people about a bad experience.

The first type is way more useful, since there is no way to tell whether one person's bad experience is typical or not.

Possibly much less common, but another type of reviewer which I fit into is "Doesn't bother to review unless a place has been consistently awesome over multiple visits, so much so that I feel they deserve my thanks".

Sure they have. 24 secs into the opening of the Siskel and Ebert TV show, have truck with logo saying "Trust Roger Ebert"

http://www.youtube.com/watch?v=tDdNVHoalJ0 (O.K., was made up for the show, but illustrates the general principle).

why is this the biggest thing we're trying to solve? can't we use these incredibly powerful algorithms to solve a real problem? there are so many restaurant recommenders out or coming soon."You have to ask yourself, are we working on the right things?"

I really wish there was a professional review company that charged restaurants for the reviews and then made the information public. Zagat's model of charging for the ratings isn't helpful because the reviews are limited to their ecosystem and don't show up in the places I read reviews (seamlessweb, google, etc)

Yelp reviews are so astroturfed the ratings are useless.

Charging the reviewed for the review makes it very hard to write a negative review, especially for subjective reviews like restaurants.

This will happen without any conscious effort:

1. Restaurants will only pay for the review if they're fairly confident it'll be good.

2. When the reviewer comes, if the manager is able able to tell who it is, he'll make sure the reviewer gets perfect service.

3. Even if the reviewer's identity is secret, the restaurant will know that the review is happening this week, and can (temporarily) improve quality for that week only. The restaurant knows more or less when the review will be happening because they paid for it (earliest possible date) and they know when it'll be published (because they'll want to know before paying).

4. When the review company grows beyond a small local area, the company will have to hire more reviewers. Most likely reviewers will review establishments close to them, that is each reviewer will have a territory he/she covers. Some reviewers will be harsher than others, so restaurants in certain areas will be ranked higher than those in others. Areas will higher rankings will grow faster (because the folks paying like higher reviews).

5. If reviewers double as salesmen, the ones who review more harshly will make fewer sales, smaller commissions, and are thus less likely to stay with the company (either from quitting or from being fired).

6. When the company's clients (i.e., the restaurants) in city A notice they overall get lower marks than those in nearby city B, they'll complain and demand to be reviewed by the same person who reviewed city B.

In short, the incentives for this company would be towards high reviews, not accurate reviews.

Reply on news.ycombinator.com