`go mod download` is for explicitly populating the cache. If you just run `go build`, the cache will be implicitly populated. You don't need to run `go mod download` before doing a build.
I opened the original issue in question due to lack of feature parity and it got quickly closed (https://github.com/golang/go/issues/26610#event-1753880630). Then after some twitter back-and-forth (https://twitter.com/sgnn7/status/1022547467466956802) the issue got reopened.