Fakespot – measure the legitimacy of reviews on popular sites
fakespot.com
fakespot.com
The two unreliable reviews I checked were two-star reviews, they were not high quality reviews, but they weren't "fake". For example one complained that they thought it was wireless, but it requires a power cord.
I checked the product's two star reviews (6% of reviews), the reviews I saw were all legit. The were long reviews, mostly complaining about the sound quality compared to the 1st gen. For reference, the top 5-star review also says sound quality was poor, until after the third firmware update.
Amazon isn't paying for fake reviews of the Echo, and no one would pay someone to purchase a $79 product (verified purchase) and give it a two-star review.
https://www.fakespot.com/product/hanger-hooks-nuolux-hanger-...
"We have noticed trends where fake review clubs/bots seed their reviewer profiles by leaving reviews on various products, specifically Amazon best selling products to create a profile that looks legitimate (it is done to fool anti-fraud systems that Amazon utilizes). Our engine detects these seed reviewer profiles and therefore penalizes them accordingly by examining their profile and our AI algorithm (one of numerous) was trained to distinguish these patterns."
I hope this is helpful. I love the website and I think it is not to the benefit of Amazon, Yelp, or TripAdvisor to do anything about fake reviews as it only helps their platforms generate more $.
Amazon is becoming jungle in truest sense where your money would be eaten away if you are not careful. Bezo keeps talking about "customer obsession" but he seems to complete ignore all of the above.
You can see pricing history (as well as set yourself price alerts) at a couple of different websites such as https://camelcamelcamel.com/
Fakespot's FAQ says the following. I think the use of machine learning is a good example of the problem: The machine needs a good data set from which to learn, which means they'd have to flag reviews in the training data as legitimate or not - but they have no way to determine that. Basically, they are training the machine to make the same judgments, of unknown, accuracy, as the humans (in fairness, I'm making some presumptions about their use of machine learning).
What criteria are used by Fakespot when analyzing reviews?
Fakespot utilizes numerous technologies to validate the authenticity of reviews.
The primary criteria is the language utilized by the reviewer, the profile of the reviewer,correlation with other reviewers data and machine learning algorithm that focuses on improving itself by detecting fraudulent reviews.
The technologies include: profile clusters, sentiment analysis, cluster correlation and artificial intelligence intertwined with these functionalities.
[0] As Richard Feynman said, "The first principle is that you must not fool yourself -- and you are the easiest person to fool."
As you pointed out, without a consistent ground truth target how could it really separate "fake" from "legit"? Perhaps they partnered with a fake review business and used their records?
Text-only based features are not fabulous, just useful (like 75% correct, using a dataset where ground truth is known). So there is / can be linguistic differences in the writing that indicate the writer is not 'truthful'. Sentiment analysis however, by itself, is abysmal - and the behavior of spammers has changed over the years so they are more knowledgeable how to craft reviews.
But other signals, like relationships to other reviewers, IP addresses, submission time of review... they have been shown to be more accurate - but not in the 90+%. Establishing who is in a spamming group seems to be reliable though, so once that is known, you can more confidently label their reviews as spam [but not 100%, of course, to be fair and objective]
I guess Fakespot takes a stab at estimating correctness and hopes the false negative rate is acceptably low. Yelp OTOH cranks things up so the false positive rate is high....
> using a dataset where ground truth is known
How is this dataset created?
We had them say ~40% of our Amazon reviews were fake/unreliable... and I'm pretty damn sure we'd know if we were faking our own reviews.
I don't know how their system works, but the false positive rate in my experience is absurd.
Amazon has virtually no competition, ergo is not incentivized whatsoever to be concerned with the integrity of their community. Further, Amazon will charge you for return shipping so, again, they couldn't care less if a positively reviewed product actually sucks. Now that I think about it, the whole system is pretty brilliant because they're double dipping: charging the customer for returns but charging the vendor for warehouse space. It's the definition of a win-win.
https://www.wsj.com/articles/on-amazon-pooled-merchandise-op...
Relevant paragraphs if you can't get past the paywall:
"As more third-party sellers have signed up to offer products through Amazon and use its order-fulfillment services, the Seattle-based giant has allowed many to pool their inventory with supposedly identical items supplied by other sellers—in essence commingling products from third-party merchants with those supplied directly to Amazon by the brands themselves.
In other words, a product ordered from a third-party seller may not have originated from that particular seller. If the bar code matches, any one that is on the shelf will do."
"FBA vendors appear to have control [...] But I still don't think the amazon customer can see if an fba vendor allows commingling"
Ten-to-one a company like Amazon buys them in a year or three. Assuming that they don't pivot to the 'extortion' business model instead.
Imagine you are appointed the new CEO of Amazon.com. Please define "enough money," even a loose ballpark estimate.
Because consumers aren't imposing sufficient costs on them for not being concerned to change that attitude. Amazon is interested in making money as efficiently as possible, period.
For example, "great place if you're a dog lover" = gang of intimidating street dogs on your doorstep. "Great eco-friendly washing machine" = one of those vibrating buckets. "Great place if you're a morning people" = loud train or church bells at 4am, or just next to a loud highway.
I've lived in some dumpy places in my life without issue so I'm not just some picky tourist expecting a Four Seasons experience in someone's guest house. But the reviews/ratings are pretty useless and Airbnb really has no reason to fix them.
There's a downside - you get reviewed yourself, as a guest. My wife had an OK experience with airbnb, then a shitty one. airbnb did nothing helpful, and the host threatened my wife with a negative review if she left one. The fact that we had documented evidence of things like
* a completely broken shower
* missing curtains in the living area, exposing your entire room to an office block across the street, with no way to walk between bathroom and bedroom without being seen by office workers day and night
* broken television
* moldy refrigerator
* and - as trivial as it sounds - 1 hanger in a closet for someone who booked a 10 day stay
airbnb hung her out to dry, no refund, no partial refund. "it must have been acceptable because you stayed". There was no guarantee of getting any refund during that time, and we were already out over $2k for just the accommodations. Horrible horrible experience, and I will not be giving them our business again any time soon.
Again, the host threatened to leave a "negative guest" review. Given that my wife would only have that one review (despite a previous airbnb stay), this has more of a damaging effect on her ability to use the service in the future than it does on the host with dozens of "fabulous/great place!" crap reviews. She declined to leave a negative review out of some odd fear, against my suggestion, but what's done is done. The airbnb team makes it pretty clear how they resolve issues, and it was not even close to being fair, from my perspective.
Of course, looking at things "objectively", yes, my wife could have come in and broken the shower, added mold to the refrigerator, broken the television and stolen the curtains just to try to wrangle a few bucks from the host. There is always that to consider, I suppose.
Some countries and cities don't have curtains. A TV service and hangers don't qualify as a valid reason to get $2k refunded, or even half of that.
We'd already been told that we'd get a negative review if she left anything, as we'd already tried to go through the 'dispute' process with airbnb.
Claiming a working TV and showing pictures of the main living area with curtains, then having neither is, at very least, deceptive. A broken shower (with video proof) apparently isn't enough to justify any dispute at all, because it was "fixed" on the 4th day. I would think, would be an admission that there was an actual problem that needed attention, and some sort of adjustment would be appreciated/welcomed.
Hangers - totally minor, but having emailed the 'hosts' earlier, they'd indicated this would be something provided, and it wasn't. I don't know how many things need to be a) promised in writing or in photos then b) not delivered before some notion of "goodwill gesture" becomes appropriate (10% off future airbnb booking? 10% refund/adjustment for poor service?) Apparently we were not even close to the threshold, and if that doesn't qualify, I have no good reason to consider airbnb for future travel needs.
If possible and necessary, deny the charge on your card.
I've been on both sides and seen all sort of claims. Disputes are always messy, AirBnb seem to read them and try to deliberate.
I have had to trash an established account with 16 great profile reviews because an agent acting for a host left a vindictive review for me.
Creating a new account takes a few minutes and I found no downside to getting places without any review history.
AirBnB isn't Facebook, people don't think it's funny that you don't have an established account.
Conversely, it doesn't work the other way around as I would be wary of booking a place with zero reviews and, I expect, most other people think the same?
Oddly though, the only place I have stayed with zero reviews (limited number of options at the time) turned out to be the best place so far?
If AirBnB would allow you to see the “review rate” of a property that would be telling. “14% of the guests of this property left a review” would mean a lot compared to “72% of the guests of this property left a review”.
On a related note, I was talking with an airbnb host yesterday who had just hosted some fairly messy guests that may or may not have broken something. He decided that not leaving a review was the best course of action. It seems that revealing the review rate could work both ways (for hosts and guests).
The place wasn't bad (by AirBnB standards) and I was surprised by this.
Would't that put us right back in the same situation? Guests and hosts will now worry about keeping their review rates above some threshold, and feel the pressure to write some kind of review. The original pressure to not leave negative reviews still exists.
I mean, I don't know if there's a solution to this. I remember a long time ago, when I bought and sold on eBay, that I rarely gave negative feedback. Just to avoid the retaliatory feedback from the other party.
What about the simple: "Would you recommend this to a friend?" And then a _short_ reason. I mean is "too much morning sun" a bad review? Nope! What it is, it is. Some people will love it. Some won't. Some won't care either way.
The point being in hiding the truth ABB is essentially inviting people to book stays that they are not going to enjoy.
The advantage of leaving a "bad"review is obvious: You might save someone from a shite experience. And if this were the stay'er community standard, that someone could be you.
Yes, inadequate light isolation in a place rented for sleeping is a significant negative feature.
Very true. I have found the places with a lot of good reviews (compared to similar nearby) tend to be the ones that allow short stays and have a high number of bed spaces despite being a one bedroom apartment.
From my own experience, you don't notice much bad stuff in a couple of days the way you do after a week or longer. Especially if among a group of friends.
A similar place that doesn't allow instant booking and only accepts long stays may be a more discerning host but get far fewer reviews in return.
That said, the lack of true clarity and transparency feels risky (to the brand). That is, if the reviews are "fake" no one - not even the property owners - can trust them. On one hand you're saying ppl leave positive reviews to bump up their reputation. On the other hand, if everyone knows the reviews are bullshit, the owners aren't going to truth them either.
Are they still reviews if they have no real truth/value?
This site has a very low bar for showing false positives. It can mark majority reviews as fake without consequences. If Amazon does something similar it will be like YouTube de-monetizing videos all over again.
> https://www.amazon.com/gp/product/B0064EKNKI
Where do they get the idea that 25% of the reviews are unreliable/deceptive? I bought it and it works just fine, which is pretty consistent with 87% of the reviews being 4+. Why should I think otherwise?
The three it flagged as suspicious for the ethernet cable above were all very short, like "Great product!" or "A++++". I'm not sure what to think of that. I write reviews (of flashlights) as a hobby and wouldn't call something that doesn't describe how the product was tested or include any technical or performance analysis a review.
On the other hand, the average buyer doesn't usually have the knowledge to write a useful technical review and just wants to report that the product met their expectations.
I mean, yes, but that wasn't the point. If the sponsored reviews seem to blend in with the normal reviews and and seem to be at least as accurate as normal reviews then how in the world are they "fake"? Where is the tangible distinction for me as a customer? (Why in the world would I avoid such products?)
> I write reviews (of flashlights) as a hobby
Funny you mention that! Flashlights seem like just about the worst-marketed products I've seen on Amazon, so they indeed definitely need great reviews. At the risk of going off-topic, do you have any tips on figuring out what the lumen output of a flashlight actually is (or even what LEDs they use)? I keep seeing Cree LEDs being claimed to output 2-3x as many lumens as they're even capable of.
Visual identification is usually the best way to tell what LED a light really uses. Here's a chart with photos of a bunch of common ones: http://budgetlightforum.com/node/26665
While the lights you find on Amazon making extreme claims about output are almost certainly lying, LED manufacturer datasheets tend to be conservative about how much power LEDs can handle. Typical mains-powered fixtures don't have the sort of heatsinking and thermal paths good flashlights do, and LED manufacturers base their limits on a service life of tens of thousands of hours. You will not use your flashlight for 50,000 hours. Overdriven LEDs aren't rare, especially in enthusiast-oriented lights.
Measuring lumens is hard because you need a way to collect all the light, or at least average it, then put that average on a sensor. Commercial integrating spheres for this purpose cost thousands of dollars. It's possible to estimate using a shoebox and a smartphone though, assuming you have a light source of known output with which to calibrate it. I wrote an app for that: https://github.com/zakwilson/ceilingbounce/
If your last flashlight used alkaline batteries and an ancient LED, or even an incandescent, anything using an LED from this decade and a Li-ion battery is going to look pretty impressive. That Li-ion cell might well be taken from a laptop battery sent in for recycling, and the charger it comes with might burn your house down, but it's still impressive to the average person.
On the other hand, I've seen a one-star review for not including a battery. It never said, or even suggested that a battery might possibly be included. My conclusion is that Amazon reviews for most kinds of products aren't useful for much more than determining if a product has an unreasonably high failure rate.
And ideally, see if you can find your charger tested here: http://www.lygte-info.dk/info/indexBatteriesAndChargers%20UK...
If it is the first one, break it so nobody else finds it and tries to use it, throw it away and replace it. The Xtar MC1 is the cheapest safe charger commonly sold in the US; they're about $5.
But some of the batteries are on there with not-great reviews... I guess I might have to replace them?
I also don't really see any particular uptick or date-clustering like you mention. There are a total of 7 reviews that say "fake" or "counterfeit" out of 177 critical reviews, out of 1382 total reviews... and I'm not really noticing a particular clustering of good reviews preceding these the most prominent critical review [1] (haven't checked the rest).
[1] https://www.amazon.com/Tera-Grand-Ethernet-Retractable-Plays...
This doesn't sound very reliable. If nothing had changed on the review, what's the basis for it changing its mind?
Anyone who uses this site really needs to do a re-analysis, because on the products I just tested I'd rate its prior ability to detect fakes (reviews and reviewers) at F.
After re-analysis I'll upgrade its ability to C.
TBH, I don't understand the point. I trust my own ability to evaluate reviews more than the unknown logic of a third party website, which may have its own interests/motives influencing its reviews of reviews.
If anyone is interested, you can find my dissertation at: https://douglas-fraser.com/datadata/
Only text based features were used, nothing involving other types of signals like reviewer name or spamming groups. I started out with 180 million Amazon reviews from 1997 onwards... but that got too big of a project, so the dataset used was a test set used in a lot of research on this topic.
I will be expanding on the research in my blog, examining other sets of text based features I never got the time to use, and things like PCA as well.
Did Novopal just buy it for him?
(Also, I wish I could plug my Amazon profile into Fakespot and see what it thinks about me.)
[1]: https://www.fakespot.com/product/novopal-baby-stroller-organ...
I rank around 6000th reviewer in FR and yet I constantly receive messages offering me to review products using this method.
Although that sounds neat, and I wish I could do that too, I suspect this would be far too helpful to actors running bogus profiles to be allowed to exist.
I ran into a software listing that had several hundred reviews, all 5 stars, but in reality is some SEO-driven bug-infested piece of crap. I wish I could take it down somehow, because of its borderline scam nature.
Review Meta does, and works very well.