Is That Review a Fake?

Fake reputation is an interesting problem space for libertarians to grapple with since reputation is their solution to so many problems. One of my ideas in this space is a bitcoin like system that maintains a public ledger of contracts you've entered into and completed as verified by a variety of sources. People could supply their public key allowing you to check their reputation history and sign contracts with their private key.

Direct link to the paper: http://aclweb.org/anthology/P/P11/P11-1032.pdf

I usually find the low-star reviews more informative. What can the 5-star reviews do other than affirm the quality claims of the reviewed book/movie/hotel/restaurant? I read 1-star reviews for a laugh and 2- and 3-star reviews to be informed by reviewers who care enough to write a review but whose opinions are not extreme.

It might be interesting that at UIC, we do a data mining course research project on this topic - http://www.cs.uic.edu/~liub/#projects. We used resellerratings.com and apart from a handpicked, there is no way of determining a pattern. There is no luxury of mechanical turks for a course project, and we are strictly forbidden from writing any reviews. I had to use the #of reviews, variance in similarity of the review text, time interval between posts, user since date, helpful review count, average rating(user/review/store),etc. and calculated the Mahalonobis distance[http://en.wikipedia.org/wiki/Mahalanobis_distance] to separate outliers as the spammers. Then with these as labeled spammers, used graph based semi-supervised learning to classify the reviewers as spammers. Its a wild open problem - and very easy to argue on both sides of any method :D

If I was writing a real review, it might look a lot like the one marked up with all the "deceptive" indicators, with the exception that I'm not a big exclamation mark user.

The grumpy old man in me wants to suggest that anyone who actually learned how to write would be marked as a fake. Is well-written English so hard to come by nowadays that it makes people suspicious when they see it online?

(Edited to add: apologies if I'm saying something that's covered in the article--couldn't get past the paywall.)

As these kinds of issues continue to bring into question the reputation of (mostly) anonymous review sites, the reliance on reviews from your social graph (I.E. folks you presumably know & trust) could continue to rise, particularly if the search engines remain unable to differentiate between the spammers and legit reviewers.

No wonder Google proactively sought out that kid who worked on the paper.

The related parent article is worth a read as well: "In a Race to Out-Rave, 5-Star Web Reviews Go for $5." (http://www.nytimes.com/2011/08/20/technology/finding-fake-re...)

The interesting point lies in how they construct their training data. In addition to producing fake reviews via mechanical turk themselves, they use a complex procedure to retrieve actual positive reviews from tripadvisor.

...which is kind of funny if you think about it: to develop a classifier that can identify real reviews, the first thing they do is create a classifier that can identify real reviews to produce a training corpus.

Then they try to approximate the output of their first (rule-based) classifier with a machine learning classifier.

personally I focus on negative reviews.

But then you have to grapple with whether or not the bad review is true or not(i.e a competitor posting it). In those cases it's best to focus on places where the reviewer's reputation can be checked.

i.e. all those review sites that let anyone review are more or less worthless. But a blog post by someone with hundreds of posts is a lot more likely to be a real story.

Shrewd (though likely just very lucky and timely) choice of topic. Looks like they can either (i) take their talents to any one of 10-15 companies that desperately need improvement in this area OR (ii) create a very promising startup and perhaps get taken out in a talent acquisition.

As businesses get more adept at gaming the online review process, the rewards for a system capable of a high "honest review" fraction will increase.

Sometimes I still cannot believe how prevalent fake reviews are.

Not too long ago a friend of mine contributed a chapter to a programming book for one of the major publishers and they made sure to tell him to ask his friends to write reviews for the book on amazon, down to detailed instructions how to fake it "properly" and in accordance with how they were trying to position it on the market - like saying that "it is great for beginners to learn X" but one should also add a negative point about how this-and-that chapter needed more clarification or more examples or images.

As if the bots wont adjust their syntax when the software becomes more comprehensive. Silly Cornell, you're up against spammers, not actual robots.