Yes, unsafe, as the name says, allows unsafe parts. But it is trivial to audit code for the usage of unsafe. Which means, everything else isn't. And it is there where the most common mistakes are made.
This is exactly the mistakes we also have in C and Rust people would do a little dance and take such bugs as argument why C is really dangerous and needs to be avoided. But rather obviously, mistakes can also happen in Rust and Rust does not "eliminate a class of errors" except when completely avoiding unsafe blocks. Maybe Rust is still more memory safe than C, I actually also believe this, but it is nowhere as safe as people like to claim and whether this is worth all the complexity is entirely unclear.
My point rather was: wherever you don't use unsafe, you are protected by the compiler from certain errors. Which I consider extremely important, that is why I am a strong proponent of memory-safe languages.
Now, if there is a question whether Rust requires you to use unsafe to often, that would be a valid technical critique of Rust, but that didn't seem to drive the Linux discussion.