Hacker News Reader

Comment by Animus7 | Hacker News Reader

I can't count how many times I've handcoded button colors, so thanks for this timesaver.

But I'm actually more interested in a deeper aspect here: what will button colors do to my conversion funnel? So here's an idea for a weekend hack: I would pay money for a PnP button A/B testing solution that automagically converged to statistically optimal colors.

View on HN

The mechanics of A/B testing math make implementing this a much less attractive proposition than you'd think it would be. I'm going to get handwavy but it will suffice for the moment: the amount of traffic you need to get statistical significance between N options grows at like O(N^2). How much of the color-space do you want to automatically converge over?

Most of my clients would struggle getting results at sufficient velocity with 10-way multivariate testing to say nothing of zomg-way multivariate testing.

Totally separate problem: Button color is far from the most salient thing you could be testing.

And if you're asking "what is?", the answer is probably your copy.

I've stopped doing minor multivariate testing in favor of A/B testing only major changes to a page.

It's just too tempting to play around with color changes all day instead of building my business.

Yup. We wasted a considerable amount of time going for silly optimizations like this when we should of been trying changing the entire copy and layout. It felt like we were doing something when in fact we were just spinning our wheels.

Me too. Major changes tend to have higher impact and provide more/faster learning. What exactly do I learn if I find that one color gives me 0.4% more conversions?

So it's not quite what you're asking for, in that you'd probably need to add something to automatically generate many color options or whatever, but http://www.mynaweb.com/ seems close and uses better math than A/B testing anyway.

Reply on news.ycombinator.com