How to A/B Test Using Javascript in Your Posterous Blog

For example, say you ran a company that had about 100 paying users, almost all of which were in the SV tech scene. A fine opportunity to use JS because they all have fast browsers and the odds that the delay between KISS Metrics or Mixpanel and them are fairly low.

If, on the other hand, you sell Bingo Card Creators to thousands of teachers around the globe the latency is going to kill you. A teacher in England will see the button (s)he wants to click on, then all of a sudden it is gone and replaced with a intro video, or, even worse, the prices have all gone up and there are different options on the pricing page.

Not only is your split test reporting the wrong winner (because of a negative bias on the B side of the test) but it can also erode the confidence visitors had in your service.