Amazon can’t end fake reviews, but its new system might drown them out | Hacker News Reader

Amazon can’t end fake reviews, but its new system might drown them out

vox.com

114 pointspabo140 comments

So here's some perspective from an Amazon seller doing 7 figures+ annually.

- On average, only 1-3% of customers review products.

- Each review is worth a lot of money, often times multiples of the product itself, and especially if you're just starting out.

- Each category in Amazon has it's own Average rating, for example, electronics typically have lower ratings because more things can go wrong and there are more usability issues vs something like kitchenware, where less things fail outright.

- If you play in a category with a certain failure rate, it is absolutely essential that you do everything you can to mitigate bad reviews as enough of them will sink your business, even if you have a great product.

- It takes 8+ 5 star reviews to counteract a 1 star review if you want to maintain a 4.5 star average which is the bar for a good product. This is extremely hard to do without manipulation.

- People who complain about fake reviews are only seeing half the problem, the other half is that legit businesses who do it the fair way can't compete. How do you launch a great product on Amazon with 0 reviews? Hope that 500 people buy it to maybe get 5 reviews? Alternatively you spend thousands on product ads hoping that enough people buy... or just succumb to the dark side and pay for reviews which is WAY cheaper.

- If you hate Amazon reviews, do your part and start reviewing the good products on Amazon. It is worth more to the seller than you think!

Here's my problem with leaving reviews:

To me, a five star review means that a product went above and beyond my expectations in some extraordinary way. I have bought products that fit that description, but not many.

Everything else, I'd give either 4 stars or a still-very-satisfactory 3 stars.

The problem is, I know these 4-and-3 star reviews actually hurt sellers, which isn't my intention at all. So I just don't leave feedback.

This is also why I don't rate Uber drivers.

> This is also why I don't rate Uber drivers.

It was here on HN a number of months ago that I learned that in the Uber world, 3 stars does not, in fact, mean "acceptable". That knowledge altered my view of all reviews, including Amazon's, in two ways:

1) Like you, it means that I'm no longer willing to give reviews/ratings. If there is no consensus on what the different numbers of stars mean, then I can't be at all comfortable that my rating will indicate to others what I intended to indicate.

2) It means that I no longer put any weight whatsoever on ratings I see from others, for the exact same reason: I can't know that what I think the rating means is at all what the rater intended it to mean.

That 3 and 4 star ratings hurt Amazon sellers underlines this problem.

Knowing how the system works and how 5 stars are expected, the way I look at Uber rating is all drivers start with 5 stars. A good enough journey gets that. But stars will be taken off if things like the car is dirty or they drive dangerously etc. It starts at 5 starts, not has to earn them.

I get why this is, but’s it’s really never been this way. Even on early eBay, if a transaction simply occurred without errors it was pure etiquette to rate them a full 5 with a comment along the lines of “A++++ SELLER! SHIPPED ON TIME, GOT EXACTLY WHAT WAS LISTED!”

Really, same thing goes for Amazon and Uber, etc. the etiquette is to start at a full rating by default and deduct based on what goes wrong. You can’t “earn” extra stars... if I order a box of batteries, and they arrive on time and the box is full, that’s a perfect transaction. 5/5, no problem. You can’t realistically expect extra batteries, or for the batteries to perfect above their rating. They’re batteries.

Did eBay ever have star ratings? As far as I can remember it's always been just "positive", "negative", and "neutral".

(However, I've never understood why so many eBay reviews will literally type "A++++++++++", we're not in school and that's just a weird thing to say.)

You can give stars for different aspects like delivery, price etc.

Thats why I turn my rating into a 1(lowest) or 5(highest).

I binary-ize any score that isn't already a yes/no, because thats what every score turns into.

This reminds me that Youtube used to have a star rating system, but as you say, it was effectively treated as thumbs up/thumbs down, so they eventually made that official.

Interestingly though it wasn't quite a simple five stars/one star dichotomy: the original blog post[0] has a dead image, but this one[1] shows that while five stars was overwhelmingly the most common rating, one star and four stars were about equal in second place.

[0] https://youtube.googleblog.com/2009/09/five-stars-dominate-r... [1] https://www.geek.com/news/google-realizes-youtube-star-ratin...

I totally should do that. I just can't.

It's wrong god damnit! Five stars should mean five stars.

Not sure I understand your problem. What prevents you from simply changing your interpretation of review scores so that it will better fit the expectations of others, instead of avoiding leaving helpful reviews altogether?

I totally agree. For five stars it would have to be exceptionally good. Something I'm very satisfied I might give four stars, which ironically drags the score down.

I recently purchased an alarm clock. When I put the clock in my bedroom, the light from the display was so bright it kept me awake. I went back to the item on Amazon and read all the reviews. One person mentioned the same issue yet still gave it four stars!

The score inflation has been plaguing many areas. this helpful chart is many years old already. It's probably a consequence of goodhart's law.

https://imgur.com/A1umWh2

i believe that's the reasoning for why many systems have devolved to boolean choices rather than multi-tier (distortions in voting cause mispresentations of the customer population's sentiments).

but boolean choices are really 3-tier systems where it's assumed that only the extremely statisfied or dissatisfied customer will vote (up or down), and the lukewarm/indifferent customer is assumed not to vote. however, that assumption very likely misrepresents the sentiments of the (majority) non-voters and thus the population as a whole.

you might address this by moving to a 3-tier system: (1) unsatisfied/bad, (2) acceptable/fine, (3) exceeded expectations/great to more accurately differentiate the non-voting/indifferent customers, but non-voters would have no incentive to suddenly voice their opinions and make the system more accurate.

you might be able to counteract that impulse by incentivizing customers to vote on every product/service delivery event (like earning points for future discounts) to lower response bias. you could also do a separate study to see how the voter/non-voter population differ, and adjust the boolean ratings accordingly.

in any case, rating systems are tricky.

Do you buy products with 3-star reviews?

I usually ignore the star ratings completely and read the text of a few reviews. This isn't a personal policy, just something I've done unconsciously. The stars don't tell me anything useful.

(Reading the reviews is by no means a good system—it's time consuming, the reviews are poorly written, and I never know what to believe. But if I ignored everything I'd feel blind.)

Before I started ignoring the ratings, I absolutely bought products with 3 star reviews, since my view was that "3 stars" means "it's fine".

And you know what? They were. I never really noticed a huge quality difference between things rated 3 stars and things rated 5. And, since I started ignoring the ratings, I've purchased things with 1 and 2 star ratings on Amazon (that were well-reviewed in other places). Those have generally been as good as the 4 and 5 star things as well -- and a couple of them were actually excellent, deserving of 5 stars rather than 2.

or just succumb to the dark side and pay for reviews which is WAY cheaper.

I thought this was what the Amazon Vine program was for -- a legitimate way for retailers to provide free products to reviewers in return for a review, and those reviews are clearly labeled as such.

Seems to work pretty well - I've bought some products that only had Vine reviews, and I've agreed with the reviewer.

Crap companies don't want Vine reviews, they want "real" fake reviews without the "I received this for free" marker.

Legitimate ones go for Vine that would have probably kinda earned it anyway.

Reviews are worth a lot to sellers.

So much so that if you leave a 1-star review, there's a good chance you'll eventually get an email asking you to remove the 1-star review in exchange for a refund, a giftcard worth the price of the product, etc.

It encourages bad behavior though; You can basically gamble on what sellers will send you that email and thus give you the product for free.

I once bought an O-ring for my blender. It's literally a 1cm-wide circle of rubber with a diameter of about 2 inches. It has 1 job - keep liquid from leaking out the bottom of the blender.

The day after I received it, the company sent me an email asking me to please please please rate my new product!

Fine! I gave it 3 stars and wrote, "It's an O-ring. It does exactly what I expected it to and nothing more."

The next day I get an email saying, "We see you gave our product a 3-star review. What can we do to improve our product? What didn't you like about our product?"

WTF? It's a goddamn O-ring! There's nothing to review beyond "it works" or "it doesn't." What the heck do you want me to say about it? No O-ring is ever going to be 5 stars. Sorry! That's just the nature of the product.

At this point, it's just harassment. Stop begging for my approval, and especially when I give you my opinion, please don't question me about why. I explained it in the review.

With the way I look at the review system, an O-Ring certainly could be worthy of 5 stars. By rating something like an O-Ring less than 5 you are implicitly saying that there are better o-rings out there, and that this one could be better. I don't see a rating for a given product as a comparison against all possible other products.

Yeah, this sort of discrepancy in the way that people think of ratings is why I no longer give or pay attention to ratings.

If I saw an O-ring with less than 4.5 stars on Amazon, I would assume it's due to piles of 1-star ratings due to it leaking, being the wrong size, etc.

If there is no conceivable way a product could be improved, it's worth 5 stars.

An O ring, just took a few minutes from your life when it malfunctioned. It took a few more mins when you had to search and place the order.

It took some more when you received it and fitted in your blender.

If it did not do its job well enough then you will have to go through this cycle all over again. I have gone through that hassle enough times to hate it, even though amazon tries to be as helpful as possible.

That there itself is worth 5 stars or atleast 4.

Beyond that not all manufacturers care about our satisfaction. If some one does and wants to improve, it should be appreciated

> there's a good chance you'll eventually get an email asking you to remove the 1-star review in exchange

Wow. I've never given an Amazon product a 1 star review, but if I did, and I got such an email, I would absolutely update my review to warn everyone that this is happening, and that the real average review for the product is likely to be lower than is shown.

I assume many people take the bait. I recently updated my review to warn people what the seller was up to after I gave a 1 star review on a defective third party Apple Watch charger. The seller had contacted me three times to get me to remove my 1 star review.

Isn’t it more of business problem that someone is hoping to stand out and become successful by selling on Amazon or any of the big platforms without creating an “unfair advantage”?

I don’t mean by gaming the system or doing something unethical, I mean in the classic business sense of creating a differentiator or in MBA speak thinking about Porter’s Five Forces.

> If you hate Amazon reviews, do your part and start reviewing the good products on Amazon. It is worth more to the seller than you think!

And who will own my reviews? I'd rather put my reviews on a more open review system.

I don't think there's anything stopping you from putting the same review in multiple systems.

Is there a system in widespread use which allows a user to accumulate reviews they have made across multiple platforms?

I may be able to trust a reviewer on product A if their opinions on B,C and D were like my own.

It seems that Amazon stores (or some franchise) could act as a good proxy for launching / improving these products - imagine running a store, and lining up products to be "test driven" in store, with all the drivers licenses and check ups one might want. I can happily imagine trying out a fairly wide range of electronics - others may prefer shoes on Thursday or Fly traps on Friday.

TBH, retail is supposed to becoming "experience" based so this might be a runner. If someone tells me Jeff's mobile number I will persuade him.

They sort of do that with their launchpad program.

https://www.amazon.com/launchpad/startups/benefits

That looks ... meh at best.

Let's say I make a Dad Bag, shoulder straps and large zips, plenty of space for nappies and milk bottles books and wipes.

I want that on blogs, instagram cool-dad accounts, so that's finding 2-300 accounts and giving them free samples, but before that I want 4.5+ stars for the darn thing on Amazon (FBA) - so how do I do that ?

Or is that the wrong way?

Where do people find paid reviewers for amazon? Seems like something they could target to close pretty easily

Upwork, fiverr, freelancer also have jobs posted which are for fake reviews, even though the job description is cleverly disguised as a a blog post or some other innocent thing.

I don't find that surprising at all. When I looked through Upwork job postings during a period of unemployment, I saw a post from a restaurant asking for a coder. The job was to create a script to cheat in an online poll from a local magazine. The poll would be used to determine which restaurant would get an award and be highlighted by the magazine, so the restaurant had an incentive to cheat. Naturally, Upwork refused to remove the post.

Facebook groups

is this Facebook that people keep talking about just some cesspool of the worst people on the internet?

Yes.

And I'm saying that as someone who considers 4chan a pretty welcoming (if loudmouthed and staunchly irrational) place.

The exception to the rule is books.

Yes, the exception is books are even worse:

https://thehustle.co/underground-world-of-kindle-ebooks

https://thehustle.co/part-2-confessions-from-the-scammy-unde...

https://thehustle.co/part-3-confessions-from-the-scammy-unde...

In my experience, making reviews easier to give causes the review quality and usefulness to go down. This happened when Netflix went from 5 stars to a simple like/dislike. I’m not sure why Amazon didn’t just block non-verified-purchaser reviews, increasing spammer costs significantly.

I suspect that Amazon reviews are going to be even less useful now. Especially given things like this:

> Amazon does not provide many specifics about how a product’s overall star rating is calculated, other than stating that it is not a simple average but instead uses “machine-learned models” that take into account factors such as how recent the rating or review is and whether it was a verified purchase or not.

I'm pretty sure that a ton of the review fraud on amazon comes through verified purchases. You refund/pay people to buy the product on amazon and leave a review in the best case, in the worst case you operate accounts buying your own stuff, and then flow the inventory back and basically pay an amazon tax for leaving good reviews.

IMO this is pretty solvable by looking at an account's purchase history too, but I don't think it's just as simple as blocking non-verified-purchase reviews.

And while you're dreaming that purchase history solves that, the scam world has long moved on.

Currently, companies pay people to buy items. They can keep those items, they just need to leave a good review. There are intermediaries who handle lots of sellers, so people buy a mixed bag of random garbage in exchange for the occasional review.

Yes, you can probably test for statistical anomalies, but I'm willing to bet that's quickly countered too - just have people buy occasional legit items so their profile is "statistically normal".

As far as I can tell, Amazon tries to fight that by keeping their ML model secret so scammers don't learn too quickly, but essentially, they're currently finding out what the Internet learned about SEOs manipulating search results.

Yes, I've seen articles where people who do this say there are a bunch of Facebook groups for exactly this purpose. Manufacturers/sellers ask people to buy their goods on the group and write good reviews. Once they get the proof of the purchase/review, they Paypal (or whatever) money to the "reviewer." I remember an article I read where some woman's house was overflowing with junk she didn't want because she was writing so many of these fake reviews. I would think Amazon could apply their awesome machine learning to figure out which accounts are pumping out these suspect reviews and perhaps de-prioritizing and maybe even rate limiting and delaying publishing of their reviews. I'd think that would do a lot to alleviate the problem.

If you stand back and think about how these schemes operate, you wouldn't actually _need_ ML to solve it.

Step 1 is getting a list of new products that are receiving a higher than expected number of 5-star reviews

Step 2, add it to a list of similar products.

Step 3, find accounts that all happen to purchase the same products in the same order.

Step 4, ban accounts and sellers.

"Step 1 is getting a list of new products that are receiving a higher than expected number of 5-star reviews"

What is higher than the expected number of 5-star reviews?

Don't those numbers vary widely depending on product category, brand, and newness of the product?

How does that work for things that fit into more than 1 product category?

> What is higher than the expected number of 5-star reviews?

That's left as an exercise for the reader ;)

But you could simplify this to new products, which if ungamed, would probably follow a predictable curve of discovery, as opposed to a games "instant 5 star" rating.

step 5: I now have a great way to boot competitors off of your marketplace.

That is a very simple algorithm! It’s also something the sort of people who do this will overcome relatively quickly. Automated or systemic attacks and mitigation is a war of escalation.

> If you stand back and think about how these schemes operate, you wouldn't actually _need_ ML to solve it.

As Machine Learning is just a fancy way of doing statistics, you will totally not need it for this.

You can do all this manually, but really you just described a form of machine learning.

I suspect solving it is hard to impossible. You can heuristic anything as suspicious at a glance but moderation at scale fails. You personally can decide "this review is bullshit" and block without issue but start flagging in a false positive or even true positive by a manipulator looking to start shit could cause considerable backlash as a futile attempt at pleasing everyone is made. Transparent and consistent rules invite gaming and opaque ones invite accusations of malfeasance and the uncertainty promotes bad feelings and bad behavior.

Worse still is that even taking a stance to not take a stance because you know it cannot be done will bring backlash as there are many who demand you take their stance, even allowing easy distribution of self chosen block and filter lists are not enough. There are many who demand the appliance experience.

There's also low scale review fraud like my sister bought a pet cam and gave it a 3 star review, and the seller started to offer escalating offers for her to remove or increase the rating from full refund, another camera, refund + free camera, and refund + money for the inconvenience.

It does bug me that scummy vendors know my email address and physical address. What's next? They send goons to my house demanding a good review or my dog gets it?

Last time I left a negative review, Amazon banned my account from making comments.

Did they think I was being paid to write a negative comment, is that such a thing? or perhaps they don’t like negative comments either. I don’t know.

Last time I left a negitive review not only did I get nothing offered, but Amazon removed the review with some nonsense reason.

As long as there is some, even small, cost to posting a review it makes review fraud much less scalable compared to zero-cost reviewing.

That's true if it costs more than it's worth.

The same applies to links and Google. Buying/renting backlinks costs money, and still pretty much everybody that wants to rank in competitive fields does it.

Perhaps they could limit refunded products to 3 star or below? Really there isn't a good reason to positively review something and take it back, if it was your fault eg if it didn't fit, then you could just not review it.

There are probably multiple ways, but often it's fine by providing the product for free in exchange for a good review.

They can for example provide a coupon that will reduce the price.

What if you refund out-of-band?

That's what usually happens. Amusingly, in the form of Amazon gift cards (so you go out and buy more products for review...)

> I’m not sure why Amazon didn’t just block non-verified-purchaser reviews, increasing spammer costs significantly.

I don't think it'd do much good. There are already large groups that subsidize verified purchases to get 5 star reviews. I read an article about it, confirmed some investigation on Facebook of on my own.

Basically someone runs a Facebook group where sellers advertise free product, promising Paypal reimbursement of the purchase price in exchange for reviews. There are at least (or were, I haven't checked recently) hundreds of Facebook groups across many languages with thousands of members each doing these activities. The reviewers are randos who like free stuff, and I don't think anyone could detect them if they only casually participate in the review scams.

I rather look at the content of the reviews than their ratings, there's been times where someone left what looks like an honest review and it's 4 / 5 but they only write positive things. I don't know what the solution is, but I definitely think more thorough reviews would help.

If you return a bad product, try and get a replacement to see if it was just bad luck, but also please do write a detailed review.

If you get a good product and it barely has reviews please review it.

someone left what looks like an honest review and it's 4 / 5 but they only write positive things

That may well be me. I don't think that "meets requirements" should merit a 5. When I do reviews, I view a bare "meets requirements" as 3 or maybe 4, depending on the type of product. I want to leave some headroom to be able to point out products that really do excel.

Another ambiguity is whether the rating is on an absolute scale, or normalized for value (it's not a perfect product, but it's super-cheap).

That makes sense, and you don't seem to be the only one, but yeah that is misleading to people like me who hadn't considered that being the reason.

I think that on Netflix's case they are not going for reviews. They want to know what you like, to suggest something you'll probably like too. Doesn't matter if it's 4 or 5 stars.

> I’m not sure why Amazon didn’t just block non-verified-purchaser reviews, increasing spammer costs significantly.

I'm not sure this would help much. I've been asked by vendors to leave a review and if it's 5 stars, I will get something in return. I think for some people this would be a "well, nobody is getting hurt, right?" decision and they'd just do it.

Amazon should provide an incentive for reporting this sort of thing.

> making reviews easier to give causes the review quality and usefulness to go down

Allow me to generalize

making X easier causes the quality and usefulness to go down

The (edit: average) Amazon star rating has been such a poor indication of the quality of a product for a very long time - at best, it's a weak indicator of which products you might check out first.

For me personally, the most valuable bit of feedback are the negative (edit: 1 and 2-star written) reviews - they are pretty much the only review content I trust. I'm looking for patterns of issues that multiple reviewers raise about the product.

The positive reviews have so little value when anyone can post a review. So many shallow positive reviews from unverified 'buyers'.

I'm not sure I agree with this. I always look for and at the 1 start reviews, and I find that to be more useful than reviews I find anywhere else.

4/5 star reviews are useless, but 2/1 star reviews reveal a great deal of useful information about a product.

A bunch of 1 star reviews of "UPS damaged my product" indicates a product that's as advertised and isn't astroturfed (much?).

No 1 star reviews, or 1 star reviews indicating misleading advertising tend to indicate astroturing.

1 star reviews indicating product failures, support failures, DOA etc... indicate to not purchase the product at all.

It's not a pure indicator, but I've found it to be more reliable than other review systems.

Am I crazy, or does the entirety of your comment except for the first sentence agree with the person you replied to?

The second and third paragraph were cut off when I replied on mobile. This was clearly my mistake.

Yes, and yes.

I think we are in agreement - I didn't word my comment well.

When I was referring to the star rating, I meant the overall average rating of the product (ie what Amazon is trying to promote now over star rating plus written reviews).

When I mentioned the negative reviews, I was referring to the 1 and 2-star written reviews. I use the negative written reviews exactly how you described - and it's what I find valuable.

I mistakenly did not see the whole OP comment when I replied. This was my fault as I'm unused to using HN on mobile and did not do my due diligence to see the full comment.

A useful heuristic that I also use, but it's good to be aware that fake reviews go in the other direction, too, of sabotaging competitors' product reviews.

plus with some of the low star reviews you can compare what they stated was wrong with your own expectations of what could go wrong with the product.

the only two considerations for me are the number of reviews and the quality of the low star reviews for the same. the dates of reviews is very useful as well, if a product doesn't have many recent good reviews it can offset the number of reviews in my view

The smart sellers realize this as well. Often the most "useful" review is also fake. Let's just create a 2 or 3 star review and list out the pros and (fake) cons and get people to upvote it.

Source: I'm an Amazon seller.

The downside of this approach is for sellers to use this as a weapon. Pay people to buy a competitor's product and give it a nasty 1 star review. It becomes an arms race.

It is also super annoying when there are reviews like "came in three days not two. 1 STAR!!!" that are not product related. This doesn't even get into the commingling of product which is the real problem.

This may help a bit with fake reviews, but it amplifies the other substantial problem -- authentic, but untrustworthy reviews. I see two type of those all the time:

1) Reviews by people who have not used the product. The tipoff for this is when the customer says something like "I just got this, it looks great! I can't wait to use it." If someone hasn't even used it, then they can't possibly give a useful review of any sort.

2) Reviews in product listings that contain multiple products, or products that have changed since the original listing. The tipoff for this is comments describing a product that is clearly not the one being listed, or is one of several different products in the same listing.

If there is an increase in reviews that are just stars without comment, it becomes impossible to root these out at all. I don't see how this would lead to more reliable reviews.

Amazon has been my wheelhouse, fulltime, for the past 7 months. I'm building a site that looks at most listings per category and determines "true" scores for each one. The site also lets your filter and sort, instantly, on a variety of attributes, like "price", "unit price", "shipping time", "recent price drop %", "used price discount", "popularity", "brand quality", etc. Spending 7 months collecting and analyzing data of several Amazon categories has been exhausting, but quite revealing.

As you can imagine, dealing with low quality products with fake reviews is a challenge -- but it turns out it's not too hard to handle, even with my dataset which is far more limited than Amazon's. Without looking at any reviews or any metadata of reviews (author, count, chronology, etc), one could filter out "impostor" products with 95%+ accuracy.

Here's a neat trick: Next time you're unsure if a product has fake reviews, click on the brand of the product and see what else they sell. If you're looking at binoculars, and that same brand also sells dog food bowls, then maybe you should reconsider.

I've concluded that Amazon really doesn't care about fake reviews -- they will show users whatever listing has the maximum Expected Value (conversion rate * revenue), per your context (search term, category, or both). Even if a product has obvious fake reviews, if there are enough other people buying it it will float to the top, and Amazon is fine with that.

> unit price

This is quite an important feature. Amazon already shows this information (sometimes at least) and yet they don't let you sort by it.

The main problem they have, I imagine, is sparse data. There are only certain fields (depending on category) which they force sellers to populate, eg: "name", "brand", etc. Item weight (which is distinct from _package_ weight), and "number of units" do not seem required, and so not many items have that information filled.

So, with sparse data, they have three choices:

1) Allow filter/sort by "unit price" and do not show the X% of listings that are missing this data -- many of which the user may actually be aware of and/or interested in.

2) Don't allow the option at all, and just rely on the fact your customer will do comparisons manually.

3) Try to derive the number of units from text cues in the product name, features, and description, then do #1.

It seems they chose #2. I'm going for #3.

At what point does the poor experience of Amazon outweigh its convenience and cause people to start shopping in physical stores again?

Or would people shift to other online retailers?

I haven't ordered from amazon in over a year. I to go to physical stores and/or order from a reputable vendor online.

Actually it's nice. If I truly want it, I'll actually go and get it, or spend the time to find a reputable vendor. The added friction ensures I don't buy random things that I thought I wanted but don't.

I've not ordered from Amazon for 3+ years now. You can buy the same products at the same price elsewhere or generally always from ebay.

The only real advantage Amazon have is prime (if you pay for it) and simple returns, but you have to deal with a drone if anything goes wrong aside from that. Most other retailers have actual people that can make decisions answering queries

For anything outdoor related, climbing, hiking, cycling. I'll always go to REI and talk with a sales person there. REI seems to hire people who have actually used the equipment they sell, I don't mind paying a small markup for the experience.

Amazon reached that point for me a number of years ago. Since Amazon reviews are largely worthless, I don't use them to decide whether or not to buy a product. I do read them, though, because they often contain useful information about specific issues and usage tips.

Amazon is fourth in my list of where I buy things from. First is a local physical store. If I can't find what I need there, then I seek out manufacturers websites, or websites of authorized retailers. If I can't find what I need there, then I check out eBay. If that doesn't work, then it's Amazon.

This changed over the last year or so, though -- before, I would go to Amazon before eBay.

I think this parallels really well with the question of how much privacy people are willing to give up for the convenience of digital services. In both cases, it seems like we are increasingly willing to give up privacy and trust in the name of convenience, but I do wonder if there is a breaking point. If there is a breaking point, is it even a system that can be reversed?

Meh, I prefer having some real reviews in an ocean of fake ones than having none at all.

Amazon has been more expensive lately and most other stores have free-shipping now too. I've definitely found myself being much more open to shopping around and purchasing elsewhere than I was 4-5 years ago.

Amazon is probably in excess of 99% perfect service for me. It feels like a step back in time when I order from other retailers.

To me a bigger problem than fake reviews is the rampant bait-and-switch. Way too often I will see a product with a 4 or 5 star rating, and find the reviews are talking about something completely different.

That, or there will be 10 different products nested under the same identifier. Not just different colors, but different products. So I have no idea which one the person it praising or calling crap. Maybe instead of Avatars next to the reviewer's name, Amazon should just attach an expandable screencap of the product page at the time the "verified purchase" was made.

> “For brands, this means the black-hat review clubs and sellers will have less impact, as fake reviews as a percentage of legit reviews should decrease.”

Well, at least they're thinking happy thoughts.

Doesn't everyone just use reviewmeta (https://reviewmeta.com/) now? They're pretty solid at what they do, the results appear to be rather accurate - and at the very least it's a decent filter.

I use Fakespot.

I think it's only a matter of time before these sites get coopted/bribed by vendors, in the same way a lot of adblockers have been.

I use it too. It's almost always the no-brand import products that add fake reviews/solicit phony glowing reviews. Most big brands have too much to lose to add fake reviews because it would be PR-damaging news.

Even so, you have to consider real reviews to be only partially-reliable because a lot of people use their products incorrectly and give excessively glowing (or negative) reviews while ignoring reality because they may feel socially-obligated to not say something negative. And then there are hundreds of millions of specialized SKUs without any reviews at all.

Hi, Saoud from Fakespot here. My co-founders and I all got scammed and duped by fake reviews and Fakespot was built to bring back trust and transparency to the e-commerce experience (and others).

We have a strict policy to remain neutral in our analyses and we do not in any way, shape or form accept bribes or copt partnerships.

Thanks for explaining the background, Sauod.

But don't you need to say that? What policies or strategies are you using to prevent this from happening beyond the shared experience of being burned and avoiding a deep sense of shame if one were to abuse their position?

We have been bootstrapped and funded by institutional investors that allows us to function freely from being in the "unnamed but big review website starting with Y" business model. Ethically speaking, I personally would never support that as it goes against our mission as a company and what we are aiming to build here.

We actually get consistent DDoS attacks which are coming from angry e-Commerce sellers that have dismal grades on Fakespot, and that is occurring ever more frequently.

I'd never heard of reviewmeta until I read your comment. I'll check it out. Thanks!

The review situation is bad enough on Amazon now that I won't buy any medium to high value product without using ReviewMeta. I find their service to be invaluable. Reading reviews and using ReviewMeta still isn't enough, but it's reasonable and much better than not using them.

Do any of these sites have an extension that can rank individual reviews for trust-worthiness based on the content + other reviews by that user?

It's always a bit of a schlep to have to remember to use another website to check review credibility.

Early adopters use such sites. A significant portion of Amazon users may not even be aware there's a problem with fake reviews.

Amazon is competing against the fake reviews with their own Vine club.

I've checked dozens of their verified & official "testers", and their five-star-rate averaged somewhere north of 98% which makes their reviews just as useless as the cheap auto-translated fakes to me.

And from our internal research at Fakespot, we concur. The positivity bias from influenced reviews such as the Vine review program and multitudes others is rampant.

Our ML based algorithms will penalize any biased reviews and show case them to our users.

When you review books on amazon, are you reviewing the content or the edition that you purchased (paper, typesetting, cover, etc)?

I wish there were two separate ratings.

I think this a constant problem with these type of rating systems.

We have also implemented one of the leading feedback-survey-customer happiness-system-things in one of our shops. The amount of negative ratings, because the shipping service provider fucked something up, is immense.

There is no way for us to really get those removed (integrity of the platform - good thing in theory) and commenthing something like "it wasn't our fault" doesn't really help either as the rating will influence the total rating one way or another.

Maybe that's cost of business, not sure.

The other problem with reviews is that they seem to be scaled according to price. A garbage product that sells cheap often gets reviewed as highly as genuinely great product that is more expensive.

Yep - and a lot of reviews are even people complaining about shipping which really is irrelevant (unless it's a packaging complaint about the product itself).

Amazon reviews are about as trustworthy as the selling points of a slimy salesman.

I've bought products with a lot of 5-star ratings which were utter garbage.

Even if you only consider "Verified Purchase" reviews, every one of them requires careful vetting for authenticity.

There are companies providing quality fake reviews by native speakers as a service, where care is taken to not be overly optimistic and provide a few negatives to sound more believable.

Even negative reviews might be fake, often mentioning a competing product that is supposed to be better.

There are plenty of valid reviews out there, but the need to constantly question the authenticity makes them almost worthless to me.

I've mostly gone back to buying from medium sized local retailers. They might have a much smaller selection and are a bit more expensive, but make up for it with a decent quality control.

It would be a shame if the reviews went away though. There are tons of garbage reviews out there, but I still find some gems out there.

Recently, I was in the market for a paint sprayer and one reviewer wrote this crazy detailed review of the sprayer he purchased, what type of paint he used to paint his cabinets, why this paint worked vs. the other ones he tried, and included numerous pictures of the process and results. It was super helpful not only in making my selection, but ultimately completing my project.

I also have found a bunch of instances where people will post tips in the reviews about how to fix or modify a product to make it perform better (or sometimes unfortunately just perform to its advertised specs). In one case I was able to repair my own refrigerator with a part purchased on Amazon and the instructions found in a review. While I'd rather have purchased it locally, that would have meant a service call and spending half the cost of the unit to get it repaired vs. a hundred or so dollars for the new circuit board from Amazon.

I think, independent of Amazon, another way to combat fake reviews is to have a modern, social-media-enabled, subscriber/patron-supported "Consumer Reports" review service that:

- doesn't accept free products or endorse any particular brands, and buys products like an individual could

- tests products thoroughly, both short-term and longer-term, including what happens within the warranty period

- reviews big-ticket items frequently enough to stay current

- offers purchasing gotchas and decision tree for selecting a product

- does the math on Total Cost of Ownership, including calculators so someone can plug in their local values (utility costs, taxes, rebates) to make the best decision

- has concise maintenance/operating advice

- starts small and looks for good products within a category

- also seeks out lesser-known, independent manufacturers who stand behind their reputations and products that people might not find on their own

- teardown and grade for repairability like iFixit

- doesn't spew blog articles that are merely copies of press releases

- compare products to those to the past to see the trends in quality, repairability, features and relative cost

The problem is that going into a store, platform or IRL, without trustworthy information leads to more arbitrary decisions. And then when there is a more reliable system for deciding what to buy, it's not that hard to have a browser extension that tags items on various platforms with recommendation badges.

There are various sites that try to present some reviews in particular genres, but the depth and breadth is usually relatively shallow and incomplete. Then there are the zillions of "review" sites that don't actually even test the products and just put manufacturer specs into a grid, and you have no idea if they have undisclosed sponsors, kickbacks or other deals.

Granted, it would be a labor-intensive proposition, so I think the best model would be a worker-owned co-op.

In the UK you have described Which. They've been around for years and getting a "best buy" rating is generally seen as a big deal (certainly it will get you sales).

Their website does have a lot of blogspam type content now, and whether you trust their "experts" is up to you. But for generic household goods, they're probably less biased than the average comparison site.

https://www.which.co.uk/

The most useful reviews will always be ones from within your own social network. Because if you have a relationship with the person, they won’t endorse something unless it reflects well on themselves.

This is why I’m bullish on protocols like SSB[1] and Iris[2].

[1]: https://handbook.scuttlebutt.nz/

[2]: https://hackernoon.com/what-is-wrong-with-the-internet-and-h...

> The new one-tap feature asks customers to select from one to five stars for a product. It’s only available to customers who have actually purchased the item from Amazon — “verified” buyers. That barrier alone creates one hurdle that will make the new rating system harder to game, since Amazon does allow written reviews from non-verified buyers.

I understand that Amazon allows non-verified buyers to leave a review, so they can have more product reviews but those non-verified reviews are the source of so much garbage.

Sure, let people post non-verified reviews, so you look like you have a lot of reviews but at least let me filter them out in my search.

Filter: 4+ stars from only verified reviews.

Hell, even 3+ stars from verified reviews would be helpful.

I know sellers can fake verified reviews by paying the consumer back for the product but that's a massively higher barrier for the manufacturer. The difference is a few cents per review VS a few dollars per review.

I've seen products with thousands of fake non-verified reviews. I'm pretty sure they couldn't afford to do the same with verified reviews.

Except negative non-verified reviews tend to be helpful, especially for sellers that constantly relist the same product under different listings.

I've bought a few products that are bad to the level of fraudulent, only to see the product relisted so that my review wouldn't be verified.

Off course this could be abused by leaving fake negative reviews for competitors.

Whack mole. Repeat.

What about making them stick, somehow?

If I leave a negative review for a product that later gets delisted, Amazon could flag the account on basis of "deleted product with poor reviews". It could then try and see if there are any substantially similar items listed a week on. If so, ban them.

Perhaps even easier, make verified negative reviews stick to the seller's account. If you delist a product with negative reviews, those just transfer over to the seller, so you see a 5-star product sold by a 1-star seller.

>But the new rating system isn’t fool-proof, since some fake-review schemers have a way to get around the “verified purchase” requirement. One popular method is to recruit buyers in private Facebook groups with a promise to refund them for their purchase via PayPal after the shoppers show proof of writing a five-star review.

I heard similar gaming techniques are used by makers of diet supplements to build up 5-star reviews. They notify customers that they will send another free bottle of supplements of they leave a 5-star review. This means that the following amazon search results example showing showing overwhelming 5-star ratings are very likely fabricated and can't be trusted:

https://www.amazon.com/s?k=lion%27s+mane+supplement

This is how you get "snake oil" products like noncompliant USB cables or defective Apple chargers into the "5-star" club that amazon buyers unwittingly trust.

Amazon should strongly incentivize their customers to report sellers who make such offers to them. This should make the game more dangerous for sellers seeking fake reviews.

If I'm somewhat "vulnerable" to "free stuff", this will not work. Couldn't care less about Amazons problems with fake reviews if I get something for free (that I kind of like).

Doesn't that just shift the game to joe jobs against the competition? There's no reason the offerers need to be closely/obviously tied to the seller. If I'm flogging Foo brand widgets and you're flogging Bar brand widgets, I offer $$$ in exchange for 5-star Bar reviews, report it and bingo, you get a kicking from Amazon.

Any such incentive by itself be prone to abuse.

Sadly, Amazon and Google reviews are mostly ignored when I am looking for a product, or service. It's so evidently clearly gamed. And Amazon's results pages with 5 of the very same product just with a different sticker/brand is also silly.

Reminds me of SEO's early days with keyword stuffing.

Slightly OT, but I noticed I've been banned from posting reviews or asking questions on Amazon. No idea why. I don't review usually and my last review was in 2017! I only noticed it a few days back because I wanted to ask a question on a product.

Does anybody know how to get out of their blacklist ? I tried emailing but get no response.

I did a quick Google to see if anyone was trying to solve this independently, and this looks pretty neat: https://thereviewindex.com/ ... Anyone have experience with it?

See reviewmeta thread https://news.ycombinator.com/item?id=22327272

What about the trade offs between privacy and the possibility of being part of a community? I mean, you have some way to identify core characteristics of the subjects whom make the reviews, and if you can't do that, how can you trust them?

I wonder what the average account age and purchase total is of genuine amazon accounts. Seems like setting some sort of age and purchase threshold to reviews could make the fake review game a lot more difficult.

One approach might be to shadowban fake accounts' reviews. They still show up as text, but they don't get counted in the average. If Amazon at T = 0 know which reviews are legitimate and which aren't, someone who consistently gives five-star reviews to shit products could have their ratings lowered in importance.

You could go all the way with this: look at my reviews, and then only show me reviews made by people with similar tastes.

A good idea. I'd wager that a high proportion of users who join these "rating review" schemes are accounts that rate 90+% of their items as 5 stars.

If you check my Amazon account, I don't even rate most of my purchases.

It's a rather low sensitivity metric but one that has some value.

I ended up just not buying stuff from amazon anymore, because I was just getting literal trash in the mail that I would have to deal with.

Yeah I cancelled my prime. Nowadays a lot of retailers will ship for free, sometimes 1 or 2 day shipping without an annual membership fee. You just have to spend a certain amount, but more and more items on amazon are 'add on' items anyway.

What kind of stuff were you buying?

The final straw was a NEW mechanical keyboard. I received a keyboard covered in perfume, that was clearly (and badly) repackaged.

Perhaps limiting the number of reviews a user can make could mitigate this somehow?

Reply on news.ycombinator.com