Maybe compilers would be default spit out binaries that run on all of them, like Apple's Rosetta or whatever it's called.
Universal binaries. Apple calls those binaries “Universal binaries”. Rosetta is more like running different arch binaries through qemu. A brief look through Google says that there was a FatELF specification created years back, but never really went anywhere. Presumably because Linux users tend to know what arch they are using.
Fat binaries would make distribution easier, but would double (or triple) the size of a binary. I doubt it would be worth the size trade off.
Docker also supports multiarch images.
Given how easy it is to JIT or BT RISC ISAs, the future is good for binary portability.
That said, I don’t know what Apple does. For example, in the main download for Big Sur, is (for example) zsh a universal binary, or are there a specific x86/M1 downloads. I haven’t looked.
/usr/bin$ du -schL .
813M total