Couple this with the fact that for a decade, the actual GUI was tied very, very heavily to the apps. So you had to use Gedit in Gnome, or whatever the KDE version of a fucking text editor is. Seemed really, really stupid. Can't we just have a relatively standard GUI text editor across platforms? Or were we all just expected to use vi and emacs?
When the windowing layers stopped moving around so much, and Greg Kroah-Hartman and co. had gotten the driver problems under control, we suddenly had a bunch of different Linuxes that were no longer compatible with each other due to packaging systems. So, now users had to download a deb, or a yum package, or a gentoo package... It was all very confusing for desktop users, especially beginners.
My quote to sum this up would be: "Why are these packages in my repository if they won't work after I install them because I'm using KDE/Gnome/Cinnamon?"
[1]: Unity took Ubuntu from being the default Linux I could put on anything, to a bloated, slow moving sack of crap. I could put Ubuntu on a 2001 laptop and be completely fine, fast enough, usable enough to do just about everything I needed.
Put Unity Ubuntu on that same machine, and it's unusable garbage. I can't even hit alt-F5 to get to another terminal because Unity slows the whole rig down to an absolute crawl. Like, can't even move the mouse, 5 minutes between keypress and action taking place. Fuck Unity. I blame Unity about 40% for killing the Linux desktop.