Fixing WordPress's code problems don't really need to break plugin compatibility. They can rewrite the core, have new features requiring doing things the new way, and have small adapters to maintain the backward compatibility.

But yes, this refactoring is going to cost and there is unfortunately not many people for whom it is a worthwhile expense. If Automattic is able to think about 10 years in the future, it might make sense for them to start investing in such a clean up.