Rust has panics as well and they appear pretty much everywhere, because the Rust stdlb made a conscious decision to panic on allocation failures (later non-panicking APIs were added, but most people dont use them, and in special, most dependencies will not use this and will panic on random occasions), and also because common operations like integer division and array indexing will panic on bugs (and also integer overflow on debug builds)
In any case OCaml has memory safety, sort of (it has data races but as described in the paper "data races bounded in time and space", data races in OCaml doesn't lead to unrestricted UB like in Rust, C, C++, and most other languages actually) (unless you opt into OCaml's unsafe constructs like Obj.magic, which is like Rust's unsafe without using unsafe {} block), because it has a GC
So when you talk about safety in the context of exceptions, you probably mean exception safety rather than memory safety. Exception safety basically means you code works even if an exception is raised. Which is really hard to assure since as you said, exceptions are everywhere
But Rust suffer from this same problem! In Rust this is called panic safety and mitigating it has taken a great deal of complexity, with things like lock poisoning, which adds overhead but limit the scope of panics in multithreaded programs, and the UnwindSafe trait, which is probably a good attempt but is ignored in most of the ecosystem. Many people think such measures are inadequate and insufficient and prefer to run programs with panic=abort, which means to just terminate the program when there is any panic. (many C++ projects disable exceptions in the same way for example)
Which is kind of unfortunate because now there are many Rust programs that are only correct if you run with panic=abort, and will break if you enable stack unwinding (which is the default)