Yes; that's the whole point. There is even a computer verified proof that (for some version of std) if you only use the standard library and don't use unsafe, then you can't write a program that exhibits unsound behavior.
Of course, if there's one bug in unsafe code, then all bets are off, but that's still a bug in the unsafe code.