And yet bounds checks were kept. Other compilers for other languages gave the option to disable some or all of the range checks, or to enable thorough validity checks. Not an all-or-nothing 'debug' toggle, but project- or file-level options for runtime checks.
On most code I worked on the bounds checks are really not having a large performance impact, especially on modern processors and it's been a very rare occasion I had to remove them, most recently converting the code to SPARK and proving the absence or runtime error. If people are OK pleasing the borrow-checker I'm sure they'll enjoy interaction with the prover (which farms most of the work to why3 and SMT solvers) to make sure their optimisation is actually safe.