I was at Super Computing 11 this past November.
I haven't done it personally. But I saw a demo at the CycleComputing booth that looked pretty slick.
They've brought up to 30,000 linux nodes. Not sure if they have an offering for windows or azure.
There are entire companies being built to solve this problem.