What would slow something like this down is not using the relevant assembly code to accelerate working with these images. A very quick glance at the convolution package, chosen on the grounds that if anything is ASM it probably is, shows that this really is "pure Go", in the sense that it doesn't use Go's ASM support. The compiler will not optimize that to speak of, certainly there's no autovectorization, so it's going to be significantly slower than an optimized program that does make correct use of vectorized operations.
Though, in theory, this can be cleaned up over time. Go does have the support built-in for fixing that in a reasonably nice way, as long as you don't mind that it has its own assembler dialect.
[1]: Not 100% related, but close enough, the part towards the end: http://www.drdobbs.com/cpp/increasing-compiler-speed-by-over...
But assuming you represent and process your image as a packed slice of rgba or something, the gc wont be in your way at all.
Source: led team that built production go image resizing and conversion services at walmartlabs.