It actually did get significantly better in Go 1.8, in early 2017. See https://golang.org/doc/go1.8 ("The overhead of deferred function calls has been reduced by about half.").
However, I think this change was mostly just an optimization for common cases with few arguments or no closures (e.g. "defer f.Close()").