I have great admiration for the amount of time and precision put into this article--as well as Amazon's relentless, cold, uncool devotion to optimization.

These meditations on A/B optimization reminded me of a genetic algorithm-based A/B testing framework mentioned not too long ago: http://ejohn.org/blog/genetic-ab-testing-with-javascript/ (I'd link to Genetify itself, but there's no homepage, just the demo)

An interesting thought: Why hasn't this caught on? Machines are good at exactly the things A/B testing aims at: measuring performance, permuting a test-matrix, and reporting on the results. This is not to say Genetify is the answer, but rather that I rarely hear of conversion optimization from developers.