I have had this concern raised, and I'm unclear how, if you adhere to that model of compatibility, that you don't effectively dispense with semver altogether and make every release a major version bump, as every change is breaking if you relied on broken or insecure behavior in earlier versions.

IMO it makes more sense to communicate the expected impact of the changes, so that downstream can read a patch bump as unlikely to cause issues.