That's technically what companies with refined processes do by using "scratch" docker images that only contain the binaries needed for the application and nothing else.

The fact that many other containers run entire OS is because it's easier to get started, or there are runtimes needed, or people just build and deploy in the same image.