Just a thought of mine: why don’t we switch fully to git? Commit signing, tag signing, Decentralize. Doesn’t that sound like a good alternativ?
Single-file archives are much easier to distribute.
Digests and signatures have standard algorithms, not unique to git. Key/identity management is the hard part, but git doesn't solve it for you (if you don't confuse git with GitHub).
The benefit to being centralized is... everything is in one place. Everything scales at once. Every update is available at the same time.
We did this back in the day using artifactory and co. to proxy NPM and a few other package managers as well as docker containers and some other things. No third party service going down could keep us from deploying.
Not everyone does it because as a solo developer or a small team, as it feels like pointless overhead.
having a decentralized, and maybe sometime unavailable, infrastructure would make more people think about the problem and maybe brings us more stable solutions than we have now.
1. Heavy dependency on Github. AKA Microsoft owns much of the golang ecosystem. Not just the source... The package distribution as well!
2. Many packages are referencing a git (short!) commit hash instead of a version. It still boggles my mind that this is an acceptable practice. Not to mention that git tags can be deleted and recreated... A pinnacle of secure package distribution practices.
3. Stuff like ambiguous imports because apparently nothing enforces proper go.mod files? They are not packages to be compiled after all, they're just repos with some conventional structure (optional)...
Mind you, this is popular production-grade software...
I think this is much worse than even node packages, let alone bundler and rubygems...
2. You complain about commit hashes while simultaneously noting that tags can be deleted and recreated. Hashes are precisely the solution to mutable tags. The "short hash" concern is a red herring; Git uses sufficient entropy that collisions are not a practical concern for dependency resolution.
As for "secure package distribution," go.sum files verify files verify consistent downloads. What additional security do you believe centralized registries provide?
3. Can you provide a concrete example of an ambiguous import you've encountered? I'm not familiar enough with Go to understand this criticism.
Exactly. This 'social phenomenon' should have been taken into account when designing a packaging system so that the language's ecosystem does not end up entirely dependent on Microsoft due to 'social reasons'.
> The go.mod file you linked references ~14 different Git hosts
Of which the non-github ones account to what... 15% of the deps in the file?
> You complain about commit hashes while simultaneously noting that tags can be deleted and recreated
Yes. Not using versions (semver) is a bad call, and having people be able to mutate the code of a version is a very bad call. Once a version has been tagged, the only viable choice must be to pull that version and push a new higher version.
> As for "secure package distribution," go.sum files verify files verify consistent downloads
Based on git's hash.
> Git uses sufficient entropy that collisions are not a practical concern
Unless crafted by an adversary? Git's sha1 hashes are not a security tool and must not be used in place of code signing.
They are also not good for versioning, as you can't deduce whether a commit introduces breaking changes. Rubygems has the ability to reference git repos. It's always a pain to update these compared to other semver deps -- you have to go to github and do a comparison between the old and new hashes to try and deduce whether bumping this will break you.
> Can you provide a concrete example of an ambiguous import you've encountered
See end of linked go.mod
> This 'social phenomenon' should have been taken into account when designing a packaging system
I'm unsure how this would be accomplished in practice without banning certain Git hosts, which seems untenable. Even Maven/Gradle ecosystems concentrate around a few major repositories (Maven Central, JCenter historically). This appears to be an inherent social dynamic rather than a solvable design problem.
> Of which the non-github ones account to what... 15% of the deps
Same question: what's the solution? Developers publish where it's easiest and most popular, creating a positive feedback loop. I don't see how package system design can prevent this.
> Not using versions (semver) is a bad call, and having people be able to mutate the code of a version is a very bad call
Agreed on both counts. However, how do we enforce immutability beyond operational controls? Even systems with "immutable" version policies ultimately rely on the registry operator honouring that policy. The only technical guarantee would be embedding content hashes alongside version numbers (which is effectively what go.sum does, albeit awkwardly).
Sidebar: how should we handle vulnerable versions? Allow pulling with warnings, or remove them entirely?
> Git's sha1 hashes are not a security tool and must not be used in place of code signing
Fair point. I was under the impression that Git had moved to SHA-256, but it seems there's no practical way to use it yet. While Git moved to a hardened SHA-1 implementation (not vulnerable to the SHAttered attack) in v2.13.0, SHA-1 remains weak for security purposes [1]. The transition to SHA-256 has been in the works for some time, but as of 2022 it appears to be a partial implementation with no support from major Git hosts [2].
What would ideal package security look like to you?
> They are also not good for versioning, as you can't deduce whether a commit introduces breaking changes
Completely agree. Repository references are useful for development and testing, but painful in production. I avoid them in published packages.
> See end of linked go.mod
Thank you, I see it now. I'm still deeply unfamiliar with Go but this feels like a legitimate criticism.
Glancing at github.com/tencentcloud/tencentcloud-sdk-go: is this import ambiguous because there's no top-level `go.mod`? If so, that feels like a significant oversight. I'm a fan of monorepos myself but I'm surprised Go doesn't have better support for them. I'll be doing some research to understand this better.
[1] https://git-scm.com/docs/hash-function-transition [2] https://lwn.net/Articles/898522/