I don't think you need to be an absolutist or only use memory safe languages but it's very obvious that we need to do a whole lot better than we actually do. Rewriting in Rust or any other language is one way to do that, but not a perfect one. I'll accept C++ when most C++ software has 1 in 50 chance of an RCE and not just a 1 in 50 chance of a known RCE. I think we can get there but we are not currently there.
Coreutils has a good track record. Ffmpeg doesn't. They should have rewritten ffmpeg in Rust, not coreutils.
There are other approaches though like formal verification.