The difference is that it's fairly easy to prevent unsafe code from occurring: a "forbid(unsafe_code)" directive prevents unsafe code from even being accepted by the compiler. In C++ there is no equivalent - good linting can prevent some obvious errors, but a lot of valid code is inherently unsafe (in the Rust sense) if used incorrectly or under the wrong assumptions.
It's also worth pointing out that most GC languages can have the same issues as a Rust program (if not more) just by virtue of having FFI. In Python, I can quite happily load a C extension and ignore all the rules. Whereas in Rust, if I want to use a C library, I still need to explicitly declare it as unsafe code.
In practice, I tend to find that I have to think more about how I approach parallel/multi-threaded code in Rust than I do in other languages - not because it's dangerous, but because the compiler doesn't allow code that would break memory safety guarantees. In some ways, that's obviously a bad thing if I'm having to think more, but in my experience, the thoughts often lead to a better design anyway. So in the end, I'm more confident in my Rust code than I am in equivalent code in a GC language.