Hacker News Reader

"Conclusion: Simplified design improved conversion by an average of 20.45%." is the most important sentence in this post.

To be honest you don't have to use Google Website Optimizer to notice that.

View on HN

It is easy to believe the things you want to believe. A/B testing is useful for convincing you of the things you don't want to believe.

Example: I'm releasing my Rails A/B testing framework this Sunday, with a case study writeup taken from my site. My site currently allows people to defer signup for the trial by means of a guest login. That sort of option is popular here. I'm told it increases usability and user engagement, right?

Without ruining the surprise: that is a testable hypothesis. ;)

A/B testing is great, no doubt. The lesson of solely simplifying is just not a great outcome of such a test. It's like driving a car for the first time and then telling everybody that it's faster than walking.

Yeah you do. I ran a test on a friend's site, convinced that a short version would outperform the long-copy version. Conversions dropped by about 30%, which would have cost him tens of thousands of dollars per year.

Reply on news.ycombinator.com