Ask HN: Please help with an AB test. (Live results in comments.) | Hacker News Reader

Ask HN: Please help with an AB test. (Live results in comments.)

theplanis.com

4 pointsnotauser8 comments

My last day at my full time job was Tuesday, now I'm working on my own projects. It's a lot of fun so far (except for testing in Internet Explorer - this works in 8, haven't tried 7 yet).

I wanted to get something up quickly to test my tool chain/environments and I thought that running a simple experiment was a great way to do that. I'm AB testing two versions of requesting e-mail addresses from visitors and I'm happy to share the results with HN.

I'll post a more detailed (anonymous) breakdown later on but you can see the initial results here: http://theplanis.com/app/abtestresults/

just as important are users who DIDNT respond :) are you collecting that information?

Good point, yes I am - It's not going into the live results but I'll put that data up too at an appropriate point.

(For the interested the short link points to a tiny Django web app that's doing both the tracking and handling AJAX e-mail uploads. Everything else is client side JavaScript.)

The first time I hit the page the request for an email address just didn't turn up at all; not sure if you're capturing that possibility. (FF 3.5.3, OSX 10.5.8)

a/b testing is a great tool, but wouldn't telling people they are participating in an a/b test trying to determine a specific question 'requesting e-mail addresses from visitors' bias the results?

I'm logging referrers so I can see how much of an effect that has - if it turns out to be significant I'll segment the results.

interesting..i await the results

After 20 hours:

310 unique visits / 22 addresses captured (7%)

50% of traffic from news.yc - and generally didn't leave an email address at all.

Users from other sources were 10 times more likely to leave an addresses if shown the pop up on page load.

Reply on news.ycombinator.com