If you base off alpine, you can get useful containers quite a lot smaller than 100MB.
One example i use is an agent i deploy to kubernetes clusters to do some security scanning. The scripts are ruby and the image clocks in at 9MB compressed https://hub.docker.com/r/raesene/kaa-agent/tags/