See here[1]. From what I've gathered, you should set GOPRIVATE appropriately, otherwise the Go Module Mirror(s) and the Go Checksum Database(s) may know that you have a private repo called “github.com/you/very-private”. It could also probably reason about your dependencies a little bit.
[1]: https://tip.golang.org/cmd/go/#hdr-Module_configuration_for_...