I stand that rust is more fun to write when you work higher level, treat it higher level language, where you'll have less control over the memory model of program. It all starts breaking down and you need to become a "rust low-level expert" when you want to work closer to the memory model (copy-free, shared memory, perhaps even custom synchronization / locking models ...). It does make sense, but in my opinion figuring out how to map your own model into rust concepts is not trivial, it requires lots of rust specific knowledge, which will take a long time to learn IMO.
When unsafe was marketed to me, I thought it was a tool I could use to escape the clutches when I'm sure what I'm doing and don't want rust to fight me, but sadly it doesn't work that way in practice, but the real way is to actually write C and call it from rust.