This is the strangest warning against A/B tests I've heard. "Be careful because you might end up producing something that is #1 internet retailer". The Amazon home page might not be as sexy as the latest whizz-bang startup, but success doesn't lie.
Why do supermarkets put the most frequently sold items at the back, when it would be so much easier for customers if they were right by the entrance?