The evolving fight against sham reviews
economist.com
economist.com
If you really hate a place, just give it some scam-looking reviews and watch it burn.
It's pretty ironic, really, that Yelp, a company famous for extorting small businesses for advertising subscriptions (with the threat of hiding positive reviews), is complaining about the validity of reviews on their site.
Their whole business model, afaict, is a shady scam against people who are too busy running their business to learn how to do SEO.
But, meh tragedy of the commons.
I wonder how much it will take until someone starts a botnet discussing products on reddit..
Some possible markers of fake reviews I look out for, completely anecdotal: - products that have high hundreds or thousands of reviews, with almost all 5's and a few 1's. - products with a large disparity between the Amazon reviews and sites that (I think) are much less likely to be targeted for fake reviews, e.g Walmart.com or bestbuy.com It should be no harder to figure out an algorithm to stop this than to stop spam email, which makes me think sites like Amazon are not really motivated to stop the practice.
If you're buying tech stuff, I find that the following approach works well:
* Find the thing you want on Newegg.
* Read a few of the five-star reviews.
* Read a few of the one-star reviews.
* Read most of the four and three-star reviews. You can use these reviews to get a sense for the actual faults with the product, and to see if the inevitable DOA complaints in the one-star reviews are a real problem, or just bad luck.
* If the product checks out, buy it from wherever you like.
* How many false positives are there? How many businesses do they unfairly penalize (and how many reviewers do they humiliate)? For example, perhaps some behavior trait common to a location or social class is misinterpreted as a fake review.
* How many false negatives are there? How many fake reviews are given extra legitimacy in the eyes of users because the algorithm gave them a pass? Maybe the algorithm weeds out only the bad fake reviewers, or a certain kind of fake reviewers. Perhaps a cottage industry of techniques to game the algorithms will arise, like SEO. I do know that decent shills know how to fool amatuer slueths; heck, I could do it: Write something long, with specifics, add a little balance to look reasonable, mirror what some others say, etc.
I'd be interested to know how Yelp and Amazon can test their algorithms. They would need a large set of proven fake reviews, and a representative set would be much more valuable. Where does one find those?
There would obviously be a conflict of interest for the people doing the reviews which is why I think it would make sense for companies to pay an intermediary who pays another firm to do the actual review. Companies that give reviews could earn "trusted" status through Amazon or Yelp (even though the are the mafia) etc.
How to know what's trustworthy? That's more difficult to establish, but someone like an Amazon has the capacity to build something like this. So would Yelp and they both have incentives to make the reviews published on their platforms believable -else we'll all believe it's just spammy astroturfing.
Still, a few honest reviews are better than a flood of unbelievable reviews because at that point it's as if there were zero reviews.
Moreover, it's my understanding most of the astroturfing is from "bought" reviews. They ought have a way to detect that (accurately), or at least give them suspicion.
- Duplicate checking (same user on different products with similar reviews, for example)
- Meta-reviews ("was this review helpful?" - can also be gamed)
- user rating averages (all highs or all lows sometimes considered a flag)
- ratio of "first product reviews" to total reviews on a per-product basis
- "Super-reviewer" status (questionable, but often a flag)
- Products with low sales ranks
- Review ring detection (IP block, post times, etc.)
- Early reviews
- Users who give high ranks, while most other reviews are low
- Positive reviews for one brand's products, and negative for others
- "Verified" purchases
A lot of these already have countermeasures, and fake reviews have already come up with counter-counter measures (like review-time staggering, using multiple IP blocks, shipping empty boxes to defeat verified purchases, etc.)
The "how to know what's trustworthy?" is the million dollar question, and it has many answers that happen to change over time. Not easy to solve.
Could they add a kind of 2FA when they ship product to the customer? Understandably, this would drive down the rate of buyers posting reviews, but could minimize astroturfing rings. In your confirmation email you get a link which can only be used to review that product, if product is subsequently returned, then weigh review less, specially if positive, if too many products are returned, remove reviews for those products.