I suppose in some ways it makes programming more difficult, but it's also going to make low-level programming safer by virtue of the fact that certain invalid programs are no longer possible to express. Rust is one of very few languages which can offer that.
(Also, having used Rust a fair amount, I don't think it's actually that difficult once you're used to it. I can think of much more complicated programming features that have been present in popular languages for decades!)
Anyone who's dealt with manual memory management already knows about these problems and rust solutions shouldn't be that foreign.
My biggest issue with rust is that it's still in early phases even considering the 1.0 release, the language is still not really that expressive and plenty of stuff still looks incredibly tedious to do.
I think 1.0 release is "here's something you can use" but it's not "here's something you'd want to use". I think I will need to wait till 2.0 so things get polished out and early adopters find all the initial design problems.
Especially regarding closures, they are really painful to use because nothing is properly inferred. You can easily improve type inference by just improving the compiler, this is almost not even a language issue. But it's a real quality of life issue.
So after 1.0 is out you will see a lot of RFCs for things like variadic generics, higher kinded types, etc. that will make someone's life easier (and remove really ugly hacks due to the lack of said things) but are completely backwards compatible.
I really want a language that can replace C++ but ATM Rust doesn't feel like it would be more productive for the things I want to do as a language and then there's the tooling/libraries/platforms.