It was a combination of the massive, massive hype train that was generated by KDE + distro's not listening to KDE's ask for 4.0 and earlier versions not being shipped that caused a massive backlash. KDE learned from the mistakes and the 4 to 5 transition and 5 to 6 transition which is in the works now are both avoiding those mistakes. But KDE did fix those bugs. Within a few releases, KDE 4 was quite usable and stable, but the damage was done.
In my opinion KDE hasn't really recovered the bad image caused by that despite the fact that its in a much better position now.