If you are using public images, then you are already giving up on optimizing image sizes anyways.

You don't have to give up public images. Here's a simple example rule - production images should all be built on Ubuntu:18.04. (With a large enough maintenance team this can change to - you must use one of these 8 images with the standard runtime stacks preconfigured).

Teams still have the flexibility of using whatever they want during the build stage, and it takes away the ambiguity of them having to decide between different OS flavors, versions, slim/Alpine, building from scratch or whatever else. In 99% of cases there will be no functional difference between them anyways, and this rule reduces the overhead of having to track down and manage an infinite number of permutations from an operations side.