This makes sense if you are doing a one-time split test, but not if you are doing ongoing tests akin to the scheme described in the article I link to below (which comes up on HN from time to time). The problem being that in these cases you never really stop testing: instead, you always assign some random proportion of incoming traffic to your various buckets based on your current statistical certainty.

http://untyped.com/untyping/2011/02/11/stop-ab-testing-and-m...

This not only has the advantage mentioned where you limit your regret for making a bad decision on too little evidence, but with some modifications to the algorithm (such as discounting old evidence from your inference) you can also deal with situations where the world in which you are running the different variations is itself changing over time, or where different types of users coming from different browsers may have different behaviors.

From the perspective of someone searching for your site, however, the results might be nearly identical: even if you have massive changes to the phrasing (having versions written in various regional dialects of English, for example), the resulting page will still largely be identical, possibly sentence-for-sentence (though not word-for-word). In this case, you will still want to get one of those variations indexed, and it might not matter which one.

As described, of course, this is both "cloaking" and not: the site is not treating GoogleBot differently than any other visiter on purpose, it is simply going to end up doing that because GoogleBot has different behavior on the site than a normal user, so it will learn and optimize (possibly somewhat randomly or uselessly) from this and end up treating that user differently than one coming from Internet Explorer (which may very well represent a different demographic of user).

It sadly does not seem like this advanced usage of testing is allowed by Google.