Hacker News Reader

Comment by gailees | Hacker News Reader

Do people actually do A/B testing on iOS apps? I feel like the two-week development cycle really hinders your ability to conduct valuable A/B tests with your users in time to actually make any change.

View on HN

Some of the more forward thinking folks (Socialcam and HauteLook for example) have rolled in house versions of these, but Apptimize is one of the first end to end solutions I've seen. But A/B testing in mobile is definitely something a lot of folks are thinking about.

We faced the same problem, and built an A/B Testing Framework at a Hackathon this month that allows you to just drop in a line of code and run all the experiments through a web interface without app updates.

The framework is not quite ready to ship yet, but we built a fully working proof of concept, and are looking to complete it and put it out there.

You can read more and watch a video demo at the website: http://fliptest.io

Take a look at http://useartisan.com/

Basically, it removes the 2-week development cycle, and you can essentially make changes in real time. I've seen it demo'ed, and it's much more impressive than the website makes it out to be.

It sounds interesting, but how do they get around the app store approval process?

You compile once, and change the app as needed. You don't need to resubmit.

You're right the app store is a barrier but people still do it. That's one of the advantages of our product, that you can start and stop experiments so you can put in as many variations as you want. Currently people just suffer through the really slow time frame.

We build in DB-driven tests into our app so we can make changes at will once we find a winner. Then we just clean up the code to remove the test in the next iteration.

Reply on news.ycombinator.com