They didn’t just have a few bugs in their browser implementation or were slow to implement new features.
Basic, crucial CSS features were broken. Microsoft didn’t add any new features to the browser about a decade, basically holding the entire web back to the standards of circa year 2001 until it was finally safe to drop support for it almost a decade later.
And furthermore, Microsoft weren’t pushing users to upgrade to the latest version, so we had to support older versions of IE long after their replacements were released.
I understand wishing that Safari would be better, but luckily the Safari team is much more on the ball than the IE team ever was.