I don't think even the Freakonomics followup is valid because there's a huge difference between telling someone to run some searches and actual search engine use.

Here's what I might type if someone asked me to try their search engine: "Miley Cyrus". The funny thing is that I'm not sure if I would like Google's "Miley Cyrus" results better than Bing's. I'm not sure I could tell the difference.

Here's the kind of stuff I actually need to work well during my normal day: "xcode 5 warning signature expiration does not match provisioning profile"

The second one isn't a real XCode warning, and that's kind of my point. I can't generate compiler warnings from my head to test Bing. I have to see them on my screen first spontaneously. This means if I'm running a Bing challenge I can't possibly simulate the real world unless I'm given my own machine and a few hours at my leisure to run into these types of things.