That is a reason for why will be impossible to fix Python (you can't assert that anything will keep their types) or in this case Rust (several features are "made" with the sole purpose to make the compiler pipeline SAD!).
You need certain discipline to develop both the language/syntax/semantic so the compiler can exploit them for speed.
---
However, I suspect it must be possible to speed up Rust and other langs massively, because you can chew gigabytes in a RDBMs in seconds. I now wonder if a "compiler database engine" could help, so you can store the lexemes, sources, inferences, etc and query them.
Also, A LOT of the work of compiler is just repeat the same steps over and over, and exist some research about save pre-computed snippets to avoid that.