I'm sorry, I don't understand. `go get` or any package manager needs to be able to fetch the code remotely. Without interacting with the external binaries would mean the Go tool would need to have custom code to retrieve the codebase and interact with the VCS metadata such as tags. Are you suggesting custom Go code for that instead of using dependencies like git?
No, git would have to be a package in the same package manager, as a build dependency. Think OS package manager, but limited to what developer needs, i.e. just software, no services, no superuser privileges, etc.
Talk about feature creep! It sounds like you want a package manager and ecosystem that can install half an OS, just so you don't have to fuss with installing git on your own...
It's going to be "half an OS" either way. But it can be nice and reliable, if you go the nix road.
More specifically, there is already a package manager and ecosystem that can install all of an OS, and you don't have to fuss with installing anything on your own. This package manager and ecosystem could subsume the problem of language-specific package managers entirely.
What manager? Can it run on Windows? Linux? Mac? If it can't do all 3, it's already failed the first requirement...