Yes, some valid code won't pass the borrow checker. But at the same time, far more invalid code doesn't pass it. Personally I see it as a tool to help me reduce possible runtime errors, and instead have them as compile time errors. This does mean that I often need to design my program around it, but that's try of basically every other language.

The few times I do have an issue I'm usually writing FFI code to interact with C, and I try to minimise that code as much as possible.