Show HN: ABBA - Beautiful, open-source A/B test calculator

thumbtack.com

51 points·birken··7 comments

Great explanations. For people, who prefer excel we have a calculator as an excel sheet: http://visualwebsiteoptimizer.com/split-testing-blog/ab-test...

We also have a blog post that explains mathematics of A/B testing: http://visualwebsiteoptimizer.com/split-testing-blog/ab-test...

I would perhaps also have added a calculation of the power of the test. But I really love that they're actually explaining the details behind it, which a lot of other A/B-test calculators out there don't do.

For people wanting the PHP code to do the calculations: http://abtester.com/calculator/

This is something like 25 lines of code.

Mixpanel also has an A/B split testing calculator: https://mixpanel.com/labs/split-test-calculator

Theirs has a nice, simple presentation which makes it very easy to use, especially for casual users. But it includes much less information (no confidence intervals and p-values are only approximate), uses much simpler statistics (z-test with no multiple testing correction can result in highly overconfident results), and doesn't describe what underlying statistics it uses. Although not as immediately accessible, I think Abba has much more to offer to anyone who's serious about A/B testing.

Perhaps lighten up the "What is ABBA" section.. it's pretty technical and I can see it scaring people off.

Upvoted for the name.

Reply on news.ycombinator.com