1. The language is designed for zero-cost abstraction, which means we have more work to do to make it compile fast. For example, any Rust compiler must solve subtyping constraints on lifetimes, regardless of the optimization level.
2. Getting an optimizing backend and a non-optimizing backend to work well is more work than just getting a non-optimizing backend to work well.
1. There are some technical issues in LLVM that prevent Rust's -O0 from being truly -O0 (LLVM's FastISel not supporting the invoke instruction being the main one here).
2. Go's compiler doesn't have much of an optimization IR at all. From what I understand, it compiles straight from the AST to Plan 9 assembly. The equivalent in Rust would be a backend that went straight from the AST to LLVM MachineInstrs (like the Baseline JIT in SpiderMonkey). Such a backend would be the ideal way to get fast whole-program compilation but would be a non-starter for optimization, so nobody has focused on it given how much work it would be. Incremental compilation would be a better use of people's time than maintaining an alternate backend, because only incremental compilation gives you algorithmic speedups (O(size of your crate) → O(size of the functions that changed since your last rebuild)).
It still takes longer to compile Rust code than many people would like. That's why it's being worked on.
It's certainly true that the Rust compiler does quite a lot of analysis (more than Go), e.g. non-trivial type inference and borrow checking. In fact, a no-op build of libcore takes 12s for me, with 5s in type checking, 0.8s in borrow checking and less than 3s interacting with LLVM (~1s of which are LLVM actually running). Turning on optimisations pushes the LLVM time out to 4s. Similarly, libstd takes 8s to build without optimisations and LLVM only runs for 1.5s (type checking itself is about the same).
(The plan to make the compiler more parallel and more incremental improve these parts without affecting the quality of the generated code at all.)