Docker's concept of base images is quite useful. You can build your application in a convenient container, then copy the resulting binary into a container with nothing else (except SSL certificates and the time zone database, for Go code anyway).

https://gist.github.com/jrockway/cceef8bb5dcef62743f8bcbc044...

I started doing this around the time we started doing vulnerability scanning and now the containers are both tiny and free of scannable security issues. I recommend that others take this approach if possible, as having too much stuff in your container increases app startup time, storage costs, and your attack surface.