A fix is not a breaking change in the API, because “breaking” refers to expected behavior (so, yes, code that relies on a bug can be broken by a fix; presumably, if you've coded to an observed behavior differing from the spec you are aware of having done so.)
The solution clearly can't be to never ever fix bugs though.
But depending on the kind of bugs (especially when they are of the "gotcha"/"UX" kind) often it's better to just create a new API version with the corrected behaviour and keep existing software apply handle the old behaviour the best they could.
But clearly for many many other kinds of bugs (security etc) we are better served with a bug fix in the old API even if that implies a possibility for breaking somebody.
The perfect simple world that a naive interpretation of semver dreams of leads to those endless streams of major.0.0 increments fueled by better safe than sorry. "You have been warned, your code might break, no promises". After all there could always be some xkcd 1172 "every change breaks someone's workflow". In an infinite universe of monkeys on typewriters someone would have set up mission critical infrastructure based on an elaborate log4jndi deploy mechanism.
On the other extreme you have something like Java that's in the process of dropping a frozen first digit ever since 1.2. Sure, this predates semver by quite some time but if we'd try to designate meaningful major.minor.patch names in hindsight we'd certainly not go exclusively by the occasional new keywords in the syntax like var but by new feature groups like generics, lambdas and the like, most of which have been introduced without invalidating any old syntax.
"We're awesome at backwards compatibility, but this change is noteworthy enough to warrant a major, better introduce some artificial incompatibility" is something that should never happen.