Google's Chief Economist Hal Varian: Computer Mediated Transactions [pdf]

From my self-interested perspective the coolest part about the modern economy is that small businesses can participate in it as much as the big guys. Totally automated fulfillment? Check. Scalable online advertising? Check. User behavior tracking? Check. A/B testing? Check. Micro-multinational 14 time zones away from freelancers and customers? Check and check.

Heck, you can do all of the above in your spare time.

For me the most interesting data point was this paragraph on page 10:

In 2008, Google ran 6,000 experiments involving web search which resulted in 450–500 changes in the system. Some of these were experiments with the user interface, some were basic changes to the algorithm (Hoff (2009)). The ads team at Google ran a similar number of experiments, tweaking everything from the background color of the ads, to the spacing between the ads and search results, to the underlying ranking algorithm.

That's over a dozen A/B tests per day. For each product line.

Greg Linden's take on the same: http://glinden.blogspot.com/2010/02/personalization-and-diff...

Prof. Varian lectured on this topic at the School of Information at UC Berkeley in October, 2009. The audio is here -

http://www.ischool.berkeley.edu/newsandevents/events/dls2009...