I find this a surprising statement -- aren't the emails the primary channel for trying to convince a free user to upgrade to paid? Why not try A/B testing to try and boost that percentage from the ~10% to something higher?
I find this a surprising statement -- aren't the emails the primary channel for trying to convince a free user to upgrade to paid? Why not try A/B testing to try and boost that percentage from the ~10% to something higher?
That said, I do appreciate the sentiment here. This is a different context entirely, but in my ecomm past a/b testing was often employed by those seeking the politically safest growth path. I considered it proactive CYA where Product Managers would prefer premature optimization over having an opinion on what's best for the user.
I witnessed what I considered an unhealthy amount of a/b testing resources spent on micro adjustments eg moving a buy button a few pixels or changing the color slightly. There was very little calculated risk taking, which can be enabled with A/B testing employed differently. There's absolutely a place for those kinds of optimizations but I think they come much later in product maturity than when many organizations actually begin using them.
Then there's an entire other discussion about understanding the math behind building a relevant test and interpreting the results.
Like you said, moving buttons and changing colors is really just a waste of time. You have to spend the time to do the analytics research, get customer feedback, etc. to first understand the problem so you can come up with a solution. Changing the color of a button doesn't solve a problem. Rather than wasting resources on doing conversion optimization the wrong way, we've decided to hold off until we can do it the right way.