Rust already does this, they have extensive testing of new releases via e.g. crater. "Run the new compiler on every public piece of code possible" is actually a very unsurprising strategy these days and communities beyond Rust use this tactic, and this is all on top of the probably ~thousands of unit tests in their suite (not that users go looking at unit tests to decide what's "valid" Rust or not.)

As someone who's invested a lot of time in PLT and even at one point had their name on a draft of the Haskell standard, the unfortunate reality is that language specifications are very often not useful to the wide majority of people, while being quite laborious to produce and maintain and work with over time. The culture of computer programming, at large, is part of this, it's not some mathematical fact, but a social one. At the end of the day, what constitutes "Rust the language" in any formal capacity doesn't matter; "Rust the language" is a social construct in that sense, a guarantee Rust will still be Rust even if some minor bugs are fixed that they let slip by. This is how most people think of it, when you get down to the nitty gritty.

I'll also say that tons of regressions happen in mature languages with backwards compatibility. Ask any Linux distro maintainer! I don't even keep track of all the weird reasons regressions happen anymore; it's like counting sand on a beach. It's not a happy fact of life, but it is what it is, and regressions do happen, no matter if your language is stable for 5 years or 50.