Hacker News Reader

Comment by Offler | Hacker News Reader

Be careful about A/B tests, from what I understand Amazon relies on the extensively and they have helped it produce its Amazon website.

Enough said.

If you have two choices one bad and another less bad you will chose the less bad, when instead you may just want to step back and ask is either choice the correct one.

View on HN

This is the strangest warning against A/B tests I've heard. "Be careful because you might end up producing something that is #1 internet retailer". The Amazon home page might not be as sexy as the latest whizz-bang startup, but success doesn't lie.

Thank you. I can't count the times I've tried to explain this to people. The purpose of amazon's site is not that visitors think it's pretty, or even easy to use. It's to get you to buy as much as possible. And there's convincing evidence that it's quite successful in fulfilling that purpose.

Why do supermarkets put the most frequently sold items at the back, when it would be so much easier for customers if they were right by the entrance?

Indeed, not stepping back can lead you (via A/B testing) to just find a local maxima in usability/uptake/etc.

I don't particularly like Amazon's site either, I think it's cluttered, but it has improved a bit. It's still not the prettiest thing, but what it does right, though, is it makes it very, very easy to buy things. And that's all it needs to do.

But you're right about not always choosing the "less bad" when there are other options you may not be considering.

The problem is though that this observation is often used as an excuse to ignore data or avoid doing small iterative experiments. In most cases, this is a mistake. In other words, you could do a lot worse than reproducing Amazon.com.

Are you suggesting Amazon's practices have led it astray somehow?

It's just a pointed way of saying that an important part of testing is knowing what to test.

Reply on news.ycombinator.com