I Made My Shed the Top Rated Restaurant on TripAdvisor | Hacker News Reader

I Made My Shed the Top Rated Restaurant on TripAdvisor

vice.com

263 pointsgrahamel59 comments

Similar to this, a few years back my then-roommates and I created a fake restaurant at our apartment, named after the nickname we gave to our kitchen. It was meant to be our own inside joke and we gave it a couple reviews ourselves.

The 4.5 star review caused us to get some walkup traffic, at which point we thought it was too real and canceled the experiment, although the page is still live if you link to it through our review history.

The best was a group of young 20 somethings who showed up and were really good-natured about the joke, so we invited them up for a beer and they dropped us a 5-star review.

>The representative adds that "most fraudsters are only interested in trying to manipulate the rankings of real businesses", so the "distinction between attempted fraud by a real business, as opposed to attempted fraud for a non-existent business, is important". To catch these people out, TripAdvisor uses "state-of-the-art technology to identify suspicious review patterns" and says, "Our community too can report suspicious activity to us." They then quote a 2015 study that found "93 percent of TripAdvisor users said they find the reviews they read to be accurate of the actual experience".

If TripAdvisor can't catch out a restaurant which is entirely a farce how can they pretend they're catching out real businesses who might be nudging their results up a bit?

There could be a difference between a human-manipulated restaurant which still has some real business and a totally fake restaurant which can be manipulated however you like.

Thinking along the lines of something I read on HN recently, using Benford's Law[1] to catch fraud in accounting data - the essential principle is that fraudsters are constrained by some unmassagable numbers, like the monthy total or some important transactions, so the figures they do change end up altering the overall distribution in some unrealistic way. But if you can generate your own dataset (and make sure it follows Bendford's), that fraud marker disappears.

TripAdvisor might be able to detect when your restaurant is behaving very different to how it's expected to, based on some past or present data - but there's less indication something has gone wrong if you just create a new restaurant out of thin air that behaves totally consistently.

1. https://en.wikipedia.org/wiki/Benford%27s_law

Or their fraud detection is just not very good.

https://en.wikipedia.org/wiki/Occam's_razor

But the question wasn't "why did TripAdvisor not catch this restaurant?", it was "why might a good fraud detection system not catch this restaurant?", because an answer to the second question is required for TripAdvisor to be able to claim that its system is any good.

> because an answer to the second question is required for TripAdvisor to be able to claim that its system is any good

Not really, the expectation of any sort of honesty in public communications, be it from the government, corporations, or individuals, is a thing of the past.

>is a thing of the past.

No, governments, corporations, and people lied just as much in the past. It's just more transparent to see these days.

The “no” isn't warranted, because the grandparent didn't say honesty which previously existed had eroded, but that the expectation of honesty had eroded. Consistent dishonesty with that becoming more transparent is one explanation of that observed effect, not a refutation of it.

I'd also disagree quite vigorously that dishonesty is perfectly constant over the last 50 years as well. You don't have to go that far back to find a time where most politicians were generally concerned with the welfare of citizens, above all other things. Now, for whatever reason, that seems to be a secondary (at best) goal on both sides of the aisle.

> You don't have to go that far back to find a time where most politicians were generally concerned with the welfare of citizens, above all other things.

It's really hard to establish what people in either the present or the past value above all other things, but certainly the idea that political science might be primarily self-interested actors and that those devoted above all to the public interest were notable exceptions is ancient rather than novel.

That’s why I just go right to the negative reviews. Usually points out right away if the place is as it seems. Really helpful when booking hotels.

You mean the negative reviews created by the competition?

Is that really a thing? Like you I also try to read the negative reviews. If they're for dumb complaints like "the waiter had an attitude" or "the food took too long to arrive" I ignore them. It's only when I see more serious problems arising in a pattern that I'll discount the restaurant.

> Is that really a thing?

Yes, businesses creating (or paying others to create) negative reviews of competitors on online sites is a thing.

Yup. This applies for every popular site with reviews.

For other funny things with reviews, I have some experience with Polish eBay/Amazon equivalent, Allegro. The following things are regular occurrences there:

- Employees of a seller creating fake accounts (and/or asking their friends and family to do this) and "buying" things with payment on delivery in order to leave 5-star reviews and positive comments. The things obviously never get sent, so it costs only as much as Allegro bills the seller per item sold.

- Clueless people leaving negative reviews because they're clueless. Like e.g. a seller I know just got a 1-star review on quality and delivery time with a comment that said basically "I got the merchandise and it's fine, but I've been asking you for the invoice for two weeks and didn't get it". The guy bought the product only a week ago, and had the invoice sent twice already.

- Sellers fucking up, getting negative reviews, then calling each of the reviewers asking them to delete the negative review, and usually bribing them with discounts or free items.

Reviews on Allegro are very high-stakes thing for sellers; just a few negative ones can mean you lose a promoted spot or get your account suspended immediately. Such situations can literally make or break a small business seller. This leads to them reviews gamed to such absurd levels that you basically can't trust them at all, if the seller is a company.

Now these are anecdotes for one site only, but given my experience on-line, I'm pretty confident they generalize to anything popular with a review system.

Lol, yep. And I'm 99% certain they are created by the review sites themselves. Our business went from having 1-3 negative reviews to literally over a 100 negative reviews(mostly couple sentences long at most) within 2 weeks, and suspiciously we got an email from the site saying "we noticed you have a large number of negative reviews - for just $3000/year you can remove all of them if you like!". We declined to pay, after which the number of negative reviews has doubled but then stayed relatively constant.

This tells me two things:

1) the whole review site business is a huge scam

2) even if we assume some of these reviews are legitimate, they don't mean anything since you can just pay to have them removed

I do the same with anything I'm purchasing, I want to understand what it's flaws are if it be a service or a product.

Sure as one of the other comments points out, it could be created by the competition - but usually looking at the like-to-dislike ratio can tell you how much you should weight the opinion of the unhappy reviews anyway. (If 100 people review positive and 1 reviews badly, chances are you can dismiss that one bad vote).

There is something to be said about people that use TripAdvisor going to a restaurant that exists and then posting non-5-star reviews. Also, TripAdvisor might actually employ undercover test eaters to check the quality of restaurants and those just never managed to book in his fake restaurant for obvious reasons.

But yeah, even if it's true what they say, they still also need to prevent fake restaurants from going up the rankings. Otherwise people will do what the author did, push their fake restaurant to the top and then open it up one day. The initial publicity boost will be enough to stay in the spotlight for a long time, especially with people falling for group think and perceiving things there to be a lot better than they actually are, just because other people supposedly thought so.

Fake restaurants aren't only a thing bored journalists create. I went to a pop-up restaurant at a vegetarian food festival. The food and drinks were so bland and terrible I can only imagine the man running the place was a scammer.

For example he was selling a "rasberry lemonade" in a beautiful glass jug with fresh rasberry and lemon slices floating at the top. It looked delicious. The only problem was the lemonade had been diluted by about 100:1 so it was effectively over-priced water.

The fake "meat" was so terrible, he might as well should've just sold the ingredients he made the meal with.

After loosing $14 on this meal I could only conclude this man just travels around with his fake vegetarian restaurant scamming people.

You are confusing “fake” with “low quality”.

No, I've been to low-quality, poorly run restaurants. This is like calling "fake news" low-quality news.

It's a funny article highlighting a cute hole in TripAdvisor's ranking system. TripAdvisor isn't being vile our misleading when they say they don't try to catch fictitious restaurants.There were no real, negative reviews of the restaurant, so it remained a perfect five with all very plausibly real reviews.

What they should worry about is detecting fake positive and negative reviews of real restaurants, which can cause businesses and diners a lot of pain and diminish the value of TripAdvisor.

Kudos to the author for "hacking" the ranking system in an unexpected way!

I wonder if they caught any fake negative reviews of the shed, jealous of its #1 spot...

Well, congratulations on gaming a system based on trust. You can try to edit Wikipedia too and make Steve Jobs chinese if you want.

For me it was interesting to see just how successful he was with relatively little effort. Holding the #1 spot for several days _seems_ like it ought to be a big accomplishment -- yet he was able to succeed without any major obstacles. This story pulled back the curtain a bit and showed just how quickly the system can break down.

I am not sure about how little the effort was he put in. Where did all the reviews came from? You don't get a #1 spot with 5 reviews...

True, but asking a bunch of your friends to write convincing reviews isn't particularly difficult.

Did you read the article?

Besides being part comedy sketch, it's also part sociological experiment and part performance art.

It wasn't about gaming a system for financial gain in the least.

Please cite where financial gain was mentioned in the previous post.

That's wasn't the point in mentioning 'financial gain", I mentioned that as an example of gaming a system for a nefarious purposes.

I mentioned this simply as contrast, that although the system was indeed gamed, it was done so with humor and without harming anybody in the process.

So you're saying making Steve Jobs Chinese on Wikipedia is a decent analogy?

Only if you legitimately convince people that Steve Jobs is Chinese.

That should be pretty easy considering he was born in Shenzhen...

How could he have gotten these great deals producing ipods when Apple was really struggling, were he not speaking Chinese perfectly?

This was a single person gaming the system as a hobby with no reward. How well do you think you can trust this system when in reality there are restaurants with a real financial incentive and the ability to invest?

This has been pointed out elsewhere, but most security is based on the principle of rational actors.

To defend against amateurs, hobbyists, and insane people working without an understandable motivation is often MORE difficult, not less.

I think the point is that if the system is based on trust, you can't necessarily trust the system.

A true entrepreneur. Bringing, sell first and build it later, to a whole new level.

Really cool story, but he never mentioned where he got the fake reviews that propelled him to #1 in the first place. There was one mention of the original “celebrity” endorsement, but what about all the reviews thereafter? Where did those come from?

At the end of the section you referenced, regarding endorsements:

> So I contact friends and acquaintances, and put them to work.

I think that's all there is to it.

> Reviews written by real people on different computers, so the anti-scammer technology TripAdvisor utilises doesn't pick up on my hoax.

It gets a little lost in the article but this and the above quote is all they did for the reviews

So he has thousands of friends and acquaintances? Or did each leave multiple reviews?

The screengrab[1] in the article show's only 96 reviews when the restaurant got to #1

[1] https://video-images.vice.com/_uncategorized/1512580232417-n...

(edited to update the image and review count)

> UPDATE 6/12/17, 17:12: After an eagle-eyed reader spotted that each mocked-up TripAdvisor screen shot contained the same number of reviews, we have edited the images so that the number of reviews match those in the actual screen shots and replaced two of the images.

You can see in the screenshot of it being the number one restaurant that it only has 83 reviews (likely the originals from friends that likely got him into the top 10,000), yet it's still #1. I'm speculating, but TripAdvisor likely ranks not just by reviews, but probably page views and other interactions that one could consider to be viewers having a positive judgment of the restaurant. In fact, you can see from the other screenshot, the page has the same 83 reviews when it was ranked #156, so it got to #1 without receiving any new reviews.

The foot/egg shot nearly made me spew my beverage onto my monitor from laughing.

I can imagine someone turning this into a heart-warming film.

Even funnier when he actually served food and people wanted to return.

>UPDATE 6/12/17, 17:12: After an eagle-eyed reader spotted that each mocked-up TripAdvisor screen shot contained the same number of reviews, we have edited the images so that the number of reviews match those in the actual screen shots and replaced two of the images.

Wait so the whole article is fake?

Google cache at #263: https://webcache.googleusercontent.com/search?q=cache:V5owX0...

No, what they are saying is that pictures that look like TripAdvisor screenshots in the article are not real screenshots of TripAdvisor site, but mockups re-created by the author. I guess this is for copyright reasons, since TripAdvisor could plausibly claim copyright on the actual content of the site, and given that the author is messing with them, it's best not to be legally exposed.

I'm astonished a new entrant was able to make any dent at all. Mostly it's impossible for newer or smaller entries to break into the top. They must have some velocity weight that The Shed exploited.

All the social review sites have become review-farming contests, which makes them unreliable on the consumer side and frustrating on the producer side, and I don't know if that can be fixed. Yelp seems to do the best (at the cost of discouraging a lot of legitimate users by making their reviews disappear), but even so is not particularly good.

I think a curated review program, perhaps with a taste-matching algorithm like Netflix, could work much better... if the economics can be worked out.

Epic. Tripadvisor is terrible for vetting places, so are google reviews, and facebook reviews. Tripadvisor is big in the UK, more reliable review sites like yelp don't have the adoption yet. Hopefully articles like this start to change that

I haven’t laughed so much at one of these since p-p-p-PowerBook [0]

0: https://forums.somethingawful.com/showthread.php?threadid=10...

I saw the pics of the 15 years old mock computer that was sent to the scammers, and the first thing that came to my mind is: if the scammers had kept it, they could make good money reselling it now as a piece of internet lore. I guess the internet has corrupted us :).

I think trust is still a very challenging problem to solve in the online realm.

Even if you can solve it, these types of things still have a very subjective nature to them.

that's not hard, they just let anyone rate stuff... just like Amazon/Google/etc does... One day, you might get sued for doing it though...

Vice is what you get when you combine journalism, capitalism, and navel-gazing college dropouts with no skills and lots of ambition. It's Tumblr for Medium. It would be beautiful, if it weren't so blatantly terrible. In a world where where startups with good ideas, talented people and lots of cash fail regularly, these witless wonders turn garbage into viral gold. Kudos to them.

I respect your bittnerness towards the world, but Vice is no different than any of the other businesses that use easily accessible , low effort content to subsidize their high effort work (for instance: Vice uses articles like “I did everything Siri told me to for a week” to subsidize their video work, Buzzfeed uses listicles to support their genuinely good longform journalism, and Comme des Garçons uses the extremely boring “Play” sublabel to subsidize their avant-garde, expensive runway shows)

Reply on news.ycombinator.com