I can see something totally opposite in my projects - particularly recompiling after a change is very quick. And cargo check often runs in less than 1s. I never got such quick incremental recompile times in any of my earlier C++ projects, particularly when using templated or header heavy libraries like STL ;)
As for the borrow checker thing, it is not broken but it's simply a limitation. In a strictly statically checked it is usually possible to run into a situation of a valid program that gets rejected by the compiler. Here this limitation comes from the compiler following a pretty simple rule - only one reference is allowed if it is mutable. This simple rule is what makes many of Rust safety guarantees possible, and I guess relaxing it conditionally might be very tricky. NLLs have already made it a bit nicer recently, do maybe it is possible to improve it further.