Good find. Here is a summary for those who don't want to dig though the Scribd document.

Here are the two versions: http://dl.dropbox.com/u/493143/ABTest.png (A = Old Page, B = New Page)

1. Doctor FootCare updated their checkout page to include a coupon code field (and a few other minor changes)

2. With the updated version, Doctor FootCare saw a 90% decrease in revenue.

3. Once Doctor FootCare took the coupon code out of the new version (B) the conversion rate of the page was 6.5% higher than the old checkout page (A). The new page without the coupon code field is not pictured.

That is a huge change, but I would not blindly assume that this result proves that the coupon code field should not be included in your checkout form. I would also like more data about the test.

"Less fields in a form is always better" is not entirely accurate. I'd agree with "Less fields in a form increases the number of people that complete the form", but this is not necessarily "better".

It does not mean that people will actually read the newsletter or complete the checkout process or come back and spend more money, those are the real metrics to track and rarely have anything to do with how many people submit the form.

Sorry to disappoint. I have done a few A/B tests in my time, but that doesn't necessarily mean I have a comparable that I can share for any element which could possibly be A/B tested.

I would also caution you that, hypothetically assuming I had a comparable here, I might be a little hesitant to say "An A/B test in June 2010 of the behavior of elementary school teachers on summer holiday ready to get their Back to School Bingo started is a great way to predict the behavior of poor startup geeks contemplating a recurring billing arrangement."

The point of A/B testing isn't to anoint someone as the local genius on design and conversion optimization. The takeaway -- over and over and over again -- is that your local genius is routinely wrong about applying their old experience or intuition to new problems.

People pay me good money for advice on things like this question. I give answers, appropriately couched as "My best guess as to how this will play out". A lot of them whiff when exposed to actual customers.