Apart from the obvious bandwidth and storage space wasted, this is also a problem when compiling an SBOM (Software Bill of Materials) and when scanning for vulnerable dependencies. There is simply no way to tell if your Go binary is vulnerable simply by looking at the go.mod/go.sum file, you have to build and analyze the binary and hope that everything went well. Same for compiling NOTICE files for license compliance. (wink wink Apache license)
This isn't an unknown or unsolved problem either, Maven solved this ages ago.
There are also some minor nitpicks, such as their absolute requirement to do path-based semantic versioning from v2, but not from v1 or v0. If you change anything, but absolutely anything that's publicly visible, you must increase the minor version of the library you are publishing or you will have a really bad time. If you want to publish both a library and an application, you have to either version them with ridiculous version numbers, or you have to split the two.