What is the go proxy even doing?

This feels like neglected software to me. It has a strong odor of a product that the owning team does not consider a major priority and takes an ‘is it working well enough?’ approach to issues rather than an ‘is it working well?’ approach.

You can kind of identify other software systems/features of other tech products that are neglected in this same way: it’s the API that flakes out, or the UI that missed the update/reskin and probably runs on neglected legacy backends, or whatever.

And before the inevitable indictment of Google engineering’s failure to consider non-Google-scale systems: that might be the root of the neglect, but it could just as easily have a dozen more mundane reasons, from “team just doesn’t like working on it/hates the code” to “some silly operational constraint makes it very hard to make the needed changes to the system”.

Like, sure, all those things are the maintainers’ (Google’s) fault, and certainly fixable given the maintainers’ resources. This isn’t a defense of Google; just a statement that the specific reasons for goproxy sucking might not be in the list of stereotypical sweeping indictments people often presume about $GOOG.

[Also posted to Lobsters: https://lobste.rs/s/ms94ja/what_is_go_proxy_even_doing#c_vz2...]

I apologize for the traffic. We clearly need to look at the “thundering herd” you are observing. That shouldn’t be happening at all.

Separately, the bug we fixed last time were about repeatedly cloning a repo in sequence even if it was unchanged, not about redundant parallel fetches. We fixed that only for Git, because Git makes it very easy to look at a branch or tag and get the tree hash, without downloading the full repo. This is exposed as the go command’s -reuse flag. The relevant Git code is at https://go.dev/src/cmd/go/internal/modfetch/codehost/git.go#... (CheckReuse).

What we need from any VCS to implement the reuse check is a cheap way to download a list of every tag and branch along with a cryptographic file tree checksum for each one, without doing a full repo clone. At the time, I convinced myself Mercurial did not support this. Perhaps I was wrong or perhaps it does now. If anyone can help us understand how that works, we could implement the -reuse flag for Mercurial too. (Any other VCSs would be great too, but Git is #1 by a very wide margin and I believe Mercurial is #2 also by a wide margin.)

Again, apologies for all the traffic, and thanks to Ted for the excellent analysis. We will look into both.

Now been three years since source hut threatened to ban Go for this behavior. For this to still be an issue is incredible.

Guess the author should send a pleading note to be added to the “Please don’t hammer my server list”.

Relevant story (2022!) https://news.ycombinator.com/item?id=31508000

I did not understand ‘hg’ and asked Claude:

The “hg” in this passage refers to Mercurial, a distributed version control system. `hg` is the command-line tool for Mercurial (just like `git` is the command for Git).

The Go project itself was historically stored in Mercurial repositories before migrating to Git. Some Go modules may still be hosted in Mercurial repos.

I got lost in the game. too bad it doesn't keep score