Semantic versioning is an illusion. It's a human-managed attempt to convey things about the surfaces and behaviors of software systems. Best case, it isn't completely misleading and a waster of everyone's time.

There is no perfection here, but the correct way to reason about this is to have schema-based systems where the surfaces and state machines are in high level representations and changes can be analyzed automatically without some human bumping the numbers.