Another possible cause can be extensions. Not that VS doesn't have its own problems of 'doing too much work on the UI thread', but there are TONS of callback points that third parties can hook into, and lots of times people do very ill-advised things on the UI thread, leading to poor responsiveness.
We monitor it actively (via PerfWatson[0], you should install it if you want your perf problems brought to the attention of Microsoft, it should have low overhead at runtime, only kicking in when the UI thread is already unresponsive), but of course with a product as large and complex as VS, and with said product being extensible, inevitably customers run into stuff we never see internally, so we rely on getting repros/data from them to understand what is going wrong.
[0] http://visualstudiogallery.msdn.microsoft.com/fa85b17d-3df2-...
Thanks for the link. I'm happy to add it.
But as you and the parent pointed out, I've also seen huge improvements in build times with VS2012.
And with VS2013 we finally get edit-n-continue on x64.