The big borrow checker hurdle comes from developers not being used to thinking in/about lifetimes. It takes some getting used to because it enforces constraints that should be there for a non-GC language, but are not enforced anywhere else. But it is a hurdle you can get over after a few weeks of feeling like an idiot. At one point it just clicks. You start to design your types and code with lifetimes in mind, and the borrow checker is not a problem anymore and feels natural - and a helpful safeguard even.
The other complex part is the lack of inheritance and the trait system, which is also foreign for many. I feel like if you had contact with ML languages, and Haskell in particular, you will have a much easier time. It takes a different approach for structuring code.
Sadly Rust is in fact getting mroe more complex all the time though, with things like async/await added and other complex features in the pipeline (like specialization, higher kinded types, generators, ...).
All of those make sense and make the language more powerful, but at the expense of simplicity.