<% ab_test('determine_the_best_page_with_numbers_to_back_you_up', ['page1', 'page2']) do |action_name| %>
render :action => action_name
<% end %>
A short primer in unconstrained, high-dimensional, non-linear
global optimization (over time!)
You can't do it.
Unconstrained and high-dimensional means that your search space is IMMENSE. Non-linear means that the optimal designs may be arbitrarily twisty, very small, fractaline and chaotic, and, most importantly, unconnected. Temporal dependence means that even if you magically find a good solution, it may fade with time.
Fortunately, we never have to do that because we use an incredible amount of prior design information (craft) to ensure that our search spaces are much smaller. This page provides an overview of that information (and even includes a call to test) which will hopefully put you somewhere in the neighborhood of a useful page and inform you of what direction to go.
If you "just had to test" then there'd be no need for art, science, literature, stock markets, language, thermodynamics, technique, craft, effort. Computers would have already solved everything (not that we would have ever invented them).
My opinion is that testing is vital because it's your only consistent, trust-worthy way to measure your value function (conversion rates). It's how you put a stick in the water to measure depth. That is only one tiny part of a complete solution no matter how important it is.
In other words, haven't you ever tried to A/B test the full page? :)
I wonder if it's harder to figure out _what_ is better about your split tests as the tests themselves become larger, though.
Nothing's wrong with starting out with some great guidelines, and then testing it to death.
Many people have an opinion about the perfect landing page based on some narrow experience. See http://www.google.com/search?q=perfect+landing+page for examples. Absorbing these guidelines provides a good starting point for any landing page optimization campaign, but only a starting point.
Claiming you have created a "perfect" landing page is simply wrong. There is no such thing as a perfect landing page. Through rigorous A/B/n & multivariate testing you can identify pages with better conversion rates and discard poor performers. It is also important to note that this optimization is not necessarily transferable to different ad messaging, user segments, products, or conversion goals.
#10 on their list, in fact, discuss A/B testing.
However, that advise alone is useless. Testing whether A or B is better is useless if both A and B don't have a good foundation.