There's a great series by Jensen Harris, Office Program Manager, about how they designed the Ribbon: https://blogs.msdn.microsoft.com/jensenh/2005/09/26/the-why-... (you can find the rest in his blog archives, unfortunately the images are gone)
It was an incredibly complex undertaking, Microsoft did many studies, had focus groups and the whole thing (Office after all, is a major business to them, worth billions of dollars).
The end result was demonstrably better for the vast majority of users (Microsoft had hard numbers to back their design decisions).
And yet if you read many of the comments on tech forums you'd think Microsoft killed the users' puppies.
https://blogs.msdn.microsoft.com/jensenh/2005/10/31/inside-d...
https://blogs.msdn.microsoft.com/jensenh/2006/01/31/flea-mar...
I wasn't kidding when I said they went through a lot of numbers while designing the ribbon (I read the whole blog archive at one point).
On top of that, the new UI is way more discoverable for new and intermediate users. And at the end of the day, beginners are let's say 10% of the users, intermediate users at various levels are probably something like 85% and power users are probably only 5% ;)
It was definitely not a good showcase for WPF.
For example, the color was a UX design decision - WPF made it possible to do theming easier, but even so, doing it instead of using stock controls actually required extra effort. But, speaking of colors and preferences - when VS 2012 went with its "black and white" color theme a couple years later, a lot of people actually demanded specifically the return of the blue VS2010 theme. Which is why it's still there today.
As for macros, it was essentially on life support for several releases prior to all that. It was essentially a separate subproduct, coming from Office originally (since it also uses VBA for scripting), but forked ages ago, and not well maintained... and it was completely broken by changing the GUI framework. So the choice was basically to rewrite or drop it at that point, and a rewrite would require substantial work - on par with the rest of the product, really. And with <1% of VS users using macros at all, it was very hard to justify a rewrite.
I would agree, though, that WPF was not a stellar framework at the time. But, ironically, VS adopting it made it better in so many ways - for example, WPF font rendering was fixed (to be less blurry) largely because of user feedback in VS2010. Perf was improved substantially, as well.
I'd rather believe their studies/methodology is flawed than that the ribbon interface makes sense. It's the most illogical and confusing interface I have ever encountered since I got my Sinclair ZX-81 as a birthday present. I still cannot use Word without googling where to find a function in the Ribbon interface, and I have no problem with any other GUI anywhere. (To be fair, I'm mostly using Linux so I'm not Word very often.)
That something I didn't realize until lately but aesthetics is hugely important. We once updated our internal issue tracker and everyone raved about how better it is now. The thing is: absolutely nothing changed for the average user besides the style. Things like more tasteful color choice, a more modern look with square corners, etc... nothing practical.
Also, from some guys at Microsoft: if you didn't change the UI, from the user point of view, you didn't do anything.
The problem with modern UIs for beginners is that too much is hidden. Panels that only appear when swiped from the side, parts that give no indication of being clickable, highly abstract icons, lack of consistency (ex: the "hamburger" menu can be anywhere, or not be present at all).
I think the underlying thing is that simple UIs sell better because you're usually presenting it to someone that can't visualize their complex workflow needs up front. The number of times I've seen requirements that are basically sales enablement features that customers would rarely use is not insignificant. Similar to focusing on usability for low-res displays, when the only time the app would be used on a low res display is when hooking it up to a projector in a demo.