I think critics tend to compare pair programming to those moments when they are most productive. One programmer in flow may be able to really crank something cool out in a day or so. He might even work 25 hours straight to do it, and really get it out there. If he spends the next week polishing, tweaking, and adding very little functionality, he doesn't include that in his comparison to pair programming. He did it in 25 hours, it took a pair of programmers three days, the pairs are slower, even though the pairs are done and have moved on to other, important stuff.
Pair programmers who work 6 hour days can finish a lot of useful functionality in a week or two. They also have social lives, get to take long lunches, go hiking on the weekends, etc., even though they are very productive. This is why I prefer it. Plus, it's totally sustainable, and it comes with a built-in training methodology.
I used to think that everyone really should start pairing. But I'm more in the camp now that people should form little organizations that pair and basically put everyone else out of business. If it's more productive, places like hashrocket are going to become the leading shops for software. Their employees are going to make twice the market rate, have social lives, good friends at work, interesting problems to work on, and relativley low stress. If it's not more productive, they'll probably eventually have trouble competing.