Using a trendy small base image doesn't make the intermediate layers any smaller.
What you're describing is what cloud native buildpacks are intended to solve: sensible, efficient, auto-updateable OCI images. I worked on them for a while and it just hardened my heart against Dockerfiles.