The ICE issue is what turned me off Rust.
I went all-in for side projects like you (this is how I typically learn a language), but after two months or so, I was spending 30-60 minutes a day trying to figure out the cause of compiler crashes, and then tweak my code so it didn't trigger them.
I've heard the ICEs have gotten much better in 1.2 (I was working in 1.0), but I haven't picked it up again.