Both of those are problems for me. The garbage collection is probably the larger one, though, since I'm sure one could add explicit, hand-tuned SIMD into any language (auto-vectorization, while useful in many contexts, just doesn't cut it here).
The main thing is that the garbage collector is a problem in time-constrained and memory-constrained environments. I can't afford to wait to clean up my large data set until the GC decides to, and I can't afford to wait the 100ms that the GC may take. Or, more accurately, I might be able to, but I just don't know, and that lack of certainty means that a GC language is not an option for me.
But that's not to say that Go suxxors! Different solutions to different problems. I'm just saying why I won't be using Go to work on the project I'm on now.