I wrote https://github.com/jwilder/docker-squash to remove things that I know I don't need in the final image such as curl, wget, temp files, various packages, etc..

I've managed to get most images to basically the size of the base image + my app.

This process is sort of the reverse of building a single binary and adding it to a minimal image. I like that approach but it's not always straightforward w/ some applications.