We can definitely go smaller than 20MB and six layers.
Here's a solution that compresses everything into a single 8.7MB layer using tar and an intermediate staging stage: https://gist.github.com/carlosonunez/b6af15062661bf9dfcb8688...
Remember, every layer needs to be pulled individually and Docker will only pull a handful of layers at a time. Having everything in a single layer takes advantage of TCP scaling windows to receive the file as quickly as the pipe can send it (and you can receive it) and requires only one TCP session handshakes instead of _n_ of them. This is important when working within low-bandwidth or flappy networks.
That said, in a real-world scenario where I care about readability and maintainability, I'd either write this in Go with gzip-tar compression in the middle (single statically-compiled binaries for the win!) or I'd just use Busybox (~5MB base image) and copy what's missing into it since that base image ships with libc.