It's because we spent some time thinking the algorithms and optimizing them, and whenever we make changes to the compiler we make sure the times remain pretty much the same (it's hard because the compiler's size grows so the times inevitably grow, at least for the compiler). And from time to time we profile and optimize further, we like speed.

I believe with time Rust can achieve a similar performance, maybe even better because they will be able to do incremental compilation (maybe, I'm not sure how will they do that with parameterized types).

There's also the thing that most of the things in Crystal are lazy: if you don't invoke a method there are no type checks to be done for it. This means that you only pay for what you use (in terms of compile speed) but also what you don't use doesn't end up being in the resulting executable.