Hacker News Reader

Comment by AlexandrB | Hacker News Reader

In my experience, making reviews easier to give causes the review quality and usefulness to go down. This happened when Netflix went from 5 stars to a simple like/dislike. I’m not sure why Amazon didn’t just block non-verified-purchaser reviews, increasing spammer costs significantly.

I suspect that Amazon reviews are going to be even less useful now. Especially given things like this:

> Amazon does not provide many specifics about how a product’s overall star rating is calculated, other than stating that it is not a simple average but instead uses “machine-learned models” that take into account factors such as how recent the rating or review is and whether it was a verified purchase or not.

View on HN

I'm pretty sure that a ton of the review fraud on amazon comes through verified purchases. You refund/pay people to buy the product on amazon and leave a review in the best case, in the worst case you operate accounts buying your own stuff, and then flow the inventory back and basically pay an amazon tax for leaving good reviews.

IMO this is pretty solvable by looking at an account's purchase history too, but I don't think it's just as simple as blocking non-verified-purchase reviews.

As long as there is some, even small, cost to posting a review it makes review fraud much less scalable compared to zero-cost reviewing.

That's true if it costs more than it's worth.

The same applies to links and Google. Buying/renting backlinks costs money, and still pretty much everybody that wants to rank in competitive fields does it.

Perhaps they could limit refunded products to 3 star or below? Really there isn't a good reason to positively review something and take it back, if it was your fault eg if it didn't fit, then you could just not review it.

What if you refund out-of-band?

That's what usually happens. Amusingly, in the form of Amazon gift cards (so you go out and buy more products for review...)

There are probably multiple ways, but often it's fine by providing the product for free in exchange for a good review.

They can for example provide a coupon that will reduce the price.

I suspect solving it is hard to impossible. You can heuristic anything as suspicious at a glance but moderation at scale fails. You personally can decide "this review is bullshit" and block without issue but start flagging in a false positive or even true positive by a manipulator looking to start shit could cause considerable backlash as a futile attempt at pleasing everyone is made. Transparent and consistent rules invite gaming and opaque ones invite accusations of malfeasance and the uncertainty promotes bad feelings and bad behavior.

Worse still is that even taking a stance to not take a stance because you know it cannot be done will bring backlash as there are many who demand you take their stance, even allowing easy distribution of self chosen block and filter lists are not enough. There are many who demand the appliance experience.

Yes, I've seen articles where people who do this say there are a bunch of Facebook groups for exactly this purpose. Manufacturers/sellers ask people to buy their goods on the group and write good reviews. Once they get the proof of the purchase/review, they Paypal (or whatever) money to the "reviewer." I remember an article I read where some woman's house was overflowing with junk she didn't want because she was writing so many of these fake reviews. I would think Amazon could apply their awesome machine learning to figure out which accounts are pumping out these suspect reviews and perhaps de-prioritizing and maybe even rate limiting and delaying publishing of their reviews. I'd think that would do a lot to alleviate the problem.

If you stand back and think about how these schemes operate, you wouldn't actually _need_ ML to solve it.

Step 1 is getting a list of new products that are receiving a higher than expected number of 5-star reviews

Step 2, add it to a list of similar products.

Step 3, find accounts that all happen to purchase the same products in the same order.

Step 4, ban accounts and sellers.

That is a very simple algorithm! It’s also something the sort of people who do this will overcome relatively quickly. Automated or systemic attacks and mitigation is a war of escalation.

> If you stand back and think about how these schemes operate, you wouldn't actually _need_ ML to solve it.

As Machine Learning is just a fancy way of doing statistics, you will totally not need it for this.

step 5: I now have a great way to boot competitors off of your marketplace.

"Step 1 is getting a list of new products that are receiving a higher than expected number of 5-star reviews"

What is higher than the expected number of 5-star reviews?

Don't those numbers vary widely depending on product category, brand, and newness of the product?

How does that work for things that fit into more than 1 product category?

> What is higher than the expected number of 5-star reviews?

That's left as an exercise for the reader ;)

But you could simplify this to new products, which if ungamed, would probably follow a predictable curve of discovery, as opposed to a games "instant 5 star" rating.

You can do all this manually, but really you just described a form of machine learning.

There's also low scale review fraud like my sister bought a pet cam and gave it a 3 star review, and the seller started to offer escalating offers for her to remove or increase the rating from full refund, another camera, refund + free camera, and refund + money for the inconvenience.

It does bug me that scummy vendors know my email address and physical address. What's next? They send goons to my house demanding a good review or my dog gets it?

Last time I left a negitive review not only did I get nothing offered, but Amazon removed the review with some nonsense reason.

Last time I left a negative review, Amazon banned my account from making comments.

Did they think I was being paid to write a negative comment, is that such a thing? or perhaps they don’t like negative comments either. I don’t know.

And while you're dreaming that purchase history solves that, the scam world has long moved on.

Currently, companies pay people to buy items. They can keep those items, they just need to leave a good review. There are intermediaries who handle lots of sellers, so people buy a mixed bag of random garbage in exchange for the occasional review.

Yes, you can probably test for statistical anomalies, but I'm willing to bet that's quickly countered too - just have people buy occasional legit items so their profile is "statistically normal".

As far as I can tell, Amazon tries to fight that by keeping their ML model secret so scammers don't learn too quickly, but essentially, they're currently finding out what the Internet learned about SEOs manipulating search results.

> I’m not sure why Amazon didn’t just block non-verified-purchaser reviews, increasing spammer costs significantly.

I don't think it'd do much good. There are already large groups that subsidize verified purchases to get 5 star reviews. I read an article about it, confirmed some investigation on Facebook of on my own.

Basically someone runs a Facebook group where sellers advertise free product, promising Paypal reimbursement of the purchase price in exchange for reviews. There are at least (or were, I haven't checked recently) hundreds of Facebook groups across many languages with thousands of members each doing these activities. The reviewers are randos who like free stuff, and I don't think anyone could detect them if they only casually participate in the review scams.

I rather look at the content of the reviews than their ratings, there's been times where someone left what looks like an honest review and it's 4 / 5 but they only write positive things. I don't know what the solution is, but I definitely think more thorough reviews would help.

If you return a bad product, try and get a replacement to see if it was just bad luck, but also please do write a detailed review.

If you get a good product and it barely has reviews please review it.

someone left what looks like an honest review and it's 4 / 5 but they only write positive things

That may well be me. I don't think that "meets requirements" should merit a 5. When I do reviews, I view a bare "meets requirements" as 3 or maybe 4, depending on the type of product. I want to leave some headroom to be able to point out products that really do excel.

Another ambiguity is whether the rating is on an absolute scale, or normalized for value (it's not a perfect product, but it's super-cheap).

That makes sense, and you don't seem to be the only one, but yeah that is misleading to people like me who hadn't considered that being the reason.

I think that on Netflix's case they are not going for reviews. They want to know what you like, to suggest something you'll probably like too. Doesn't matter if it's 4 or 5 stars.

> I’m not sure why Amazon didn’t just block non-verified-purchaser reviews, increasing spammer costs significantly.

I'm not sure this would help much. I've been asked by vendors to leave a review and if it's 5 stars, I will get something in return. I think for some people this would be a "well, nobody is getting hurt, right?" decision and they'd just do it.

Amazon should provide an incentive for reporting this sort of thing.

> making reviews easier to give causes the review quality and usefulness to go down

Allow me to generalize

making X easier causes the quality and usefulness to go down

Reply on news.ycombinator.com