Sock Puppets. Online Reviews: Completely Worthless or Only Mostly Worthless?
techcrunch.com
techcrunch.com
And on the flipside, I've never bought something because of good reviews that all turned out to be lies when I had the product in my hands a couple of days later.
It seems that scores follow an interesting distribution, people mostly review things they love or hate. People rarely seem to want to take the time to point out something is just ok.
You see this bimodal distribution for Yelp restaurant reviews. Reading through the reviews I find the reasoning of a lot of reviewers quite spurious. I think it would be an useful application of machine learning to determine the validity of claims and generate a real score, an automatic Zagat-machine.
It's worse for complicated products, if I was a manufacturer I'd want to be able to rebut them. For example, I was looking to buy a coffee grinder and the reviews were all over the place. It's obvious that a lot didn't read the instructions or had higher expectations than a product at that price point would provide. People complain that a $30 item is made mainly of plastic.
One of the advantages of Amazon is the sheer quantity of reviews for most items, the outliers--whether positive or negative--get lost in the noise. It does penalize products with only a couple of reviews, it starts to make you nervous and reconsider the purchase.
This really is the tyranny of choice, you waste a lot of time looking for the "perfect" product, when traditionally you'd just walk into your local store, make a decision based on imperfect knowledge, and probably be happy with what you ended up buying.
A good example is my recent purchase of a nail brush from Amazon. It's a sub-$10 item that in the past I'd have just walked into my local drugstore and picked up the cheapest out of whatever they had in stock. The category[1] has 843 products that I wasted a good hour deciding on the "best" to purchase.
[1] http://www.amazon.com/s/?url=node=11063531&rh=n:3760911,...
You can rebut. I mean, for a long time the first hit for 'prgmr.com' was a thread on webhostingtalk[1] where someone was complaining that I hadn't set them up and that I had problems. Fair complaints, on the other hand, at the time, five dollars was an unheard of price for what I was providing (It's no longer an unheard-of price. Moore's law abides.) I responded and eh, I think I ended up coming out looking okay.
I guess the thing is, acknowledging your mistakes, yes, is very unprofessional, but really? I think people are sick of dealing with professionals. Showing some humanity and some humility (even when it's the 'false humility' we all put on so that our arrogance is less irritating to others.) can help you a lot.
On the other hand, on that same website, I'm trying to sell burstable 10G cogent ports[2] for a price that is currently unheard of, and I've gotten no bites, so It's possible that people prefer professionals for expensive things and humans for cheap things.
Note, though, (and this is the 'false humility' I describe) you don't want to directly say "you are too dumb to use the product" - even if they are. I mean, you want to explain the proper use of the product, and you want to back out the deals (refunds, usually) with the people that are unwilling or unable to do it correctly. Saying things here like 'Oh, I should improve this documentation here' or 'oh, I should link to this other documentation over there' is also good, especially if you really can improve that documentation. Personally, I think having a reputation for being not particularly easy to use is very good. You don't want to deal with the people that want handholding, especially when your product is as difficult to use safely as a Linux server.
[1]http://www.webhostingtalk.com/archive/index.php/t-541344.htm...
Techcrunch is a "publication" I've always found questionable and wondered how much sock-puppetry was hiding under the covers. Note I have no hard evidence, just a feel.
Sock puppets complaining about sock puppets... Go figure.
Of course, that's exactly what a sock puppet would say. Hmm. Is there a sock-puppet Turing test?
You could make millions! :)
Disprove it.
Seriously, though. Every source with a grain of salt.[1,2]
______________
[1] http://paulgraham.com/submarine.html: "If anyone is dishonest, it's...reporters. The main reason PR firms exist is that reporters are lazy. Or, to put it more nicely, overworked. Really they ought to be out there digging up stories for themselves. But it's so tempting to sit in their offices and let PR firms bring the stories to them."
[2] That's not a knock on this submission. If anything, just an application what we learned inside (ie,its principles). Or, as the author un-ironically notes, "Even the honest reviews are mostly written by people who are stupid...". That sounds fair and objective, right?