For Rust safety (meaning no UB) most definitely is a property of the language. If a module does not contain unsafe and the modules it uses that do contain unsafe are implemented soundly then there is no UB.
In C UB is a part of the language.
Also "If .. are implemented soundly" sounds harmless but simply means there is no safety guarantee (in contrast to Fil-C or formally verified C, for example). It relies on best-effort manual review. (but even without "unsafe" use anywhere, there are various issues in Rust's type system which would still allow UB but I agree that this is not that critical)
In C UB is part of the ISO language specification, but not necessarily part of a specific implementation of ISO C. If you argue that the ISO spec matters so much, I like to point out that Rust does not even have one, so from this perspective it is completely UB.
Don't those also depend on implementations being sound? Fil-C has its own unsafe implementation, formal verification tools have their trusted kernels, it's turtles all the way down.
Does make me wonder how easy porting Fil-C to Rust/Zig/etc. would be. IIRC most of the work is done via LLVM pass(es?) and changes to frontends were relatively minor, so it might be an interesting alternative to MIRI/ASan.