Hacker News Reader

Comment by makeworld | Hacker News Reader

This is why I like what Go does, where you're downloading from Git directly (optionally proxied through Google, yes)

I'm not sure what is meant by "downloading from Git", I assume you mean downloading from Github. And Github is far less secure than what crates.io does, because crates.io is immutable (once published, uploaders can't change anything without opening a support ticket which will get rejected if they don't have a good reason), whereas Github history is trivially rewriteable. This means that if you rely on "v1.2.3" of a library from crates.io, that's always going to give everyone the same code; conversely, relying on a git tag of "v1.2.3" from a random Github repo could be anything at any point.

The goproxy server makes it somewhat immutable for go too. Once they have a version cached they will never delete it. You can only supercede it with a new version and mark the old version as bad.

TBH, "somewhat immutable" is not "immutable". The Go approach aids to limit the effects of an attempted attach where you're misled into building your project with different code than originally intended, but does nothing to guarantee continuity over time of dependencies being available. For that you have to rely on vendoring.

The crates.io approach instead defends you both from a dependency changing silently and from it disappearing from one day to the next, without having to deal with vendoring.

As kibween noted above, crates.io can be changed if there's a "good reason" so it's not truly immutable either.

Go does not delete modules from the module proxy except for copyright/legal reasons (I suspect crates.io would delete for these reasons too), and additionally the checksums of all modules are published in a tamper-proof transparency log (https://sum.golang.org/) so if they did alter or delete a module, it would be detected.

The Go module proxy doesn't retain modules forever, as documented in the link there. And in the case where someone alters the underlying git repo by attaching a previously-used tag to a new commit, one of the following scenarios must then result:

1. The proxy delivers the changed code while issuing a warning to the end user, which if the warning is overlooked would mean that the checksum achieved nothing.

2. The proxy ignores the changed code while delivering the old code and a warning to the user, which would introduce the same concern raised here that the code that gets delivered and the code listed on Github have no requirement to be the same.

3. The proxy refuses to deliver any code and issues a warning to the user, which would mean that anyone can effectively remove their code from the proxy by simply changing the tag to something else.

I would be interested to know which one Go actually goes with, because none of these are ideal.

In general, the Go module proxy retains modules forever, for the explicit purpose of not breaking builds (https://sum.golang.org/#faq-retract-version). crates.io can delete content also. I don't think there is much difference between crates.io and the Go module proxy in this regard - they both aim to keep source code forever to avoid breaking builds, but will delete content if there's a good reason.

If the Git tag changes, the proxy returns the original code. There is no warning that the Git tag has changed, but it can't reliably detect this anyways because the Git repository could be returning different content to different clients. I don't think there is much difference between crates.io and the Go module proxy in this regard - in neither ecosystem can you assume the Git repo matches what the packaging tool downloads. (I pointed this out here: https://news.ycombinator.com/item?id=40699948)

Where Go is different is that it provides assurance that the module proxy is providing the same code to everyone, eliminating the module proxy as a potential source of compromise. It also ensures that people who disable the module proxy and fetch code directly using the go command get the same code that the module proxy has. To reiterate, this does not help with doing code audits of Git repos - you have to either audit the code in the module proxy, or compute the checksum of the Git repo to make sure it matches the sumdb.

Go modules can be hosted in any Git repository. The Go toolchain also keeps hashes of the selected tag so if you've reviewed it once it will never change without you explicitly giving it the ok.

> giving it the ok

You mean giving it the Go ahead? ;)

Also any svn/hg repository afaik

That’s true, except that the lockfile records revision as a commit sha.

https://github.com/apex/up-examples/blob/master/oss/golang-g...

> conversely, relying on a git tag of "v1.2.3" from a random Github repo could be anything at any point.

I don't know of a single modern build tool that can do this but doesn't require or record this information specifically? Maybe the earlier versions of Go? (I know they've gone through a few changes in module/import strategies.)

It’s interesting you even have to point this out. Maven solved this and other problems literally decades ago but the repository packaging wheel keeps getting reinvented. For example, here’s the page on Maven Central’s immutability policy:

https://central.sonatype.org/publish/requirements/immutabili...

Meanwhile the Python Package Index has recently dropped official support for PGP signatures: https://blog.pypi.org/posts/2023-05-23-removing-pgp/ -- apparently due to too few useful/verifiable signatures, as per https://blog.yossarian.net/2023/05/21/PGP-signatures-on-PyPI...

The problem is that if you clone the Git repository, or view it on GitHub, you have no assurance that you're seeing the same code that the go command or the Go module proxy saw. The author of a malicious module could change the Git tag to point to a different, benign, commit after the Go module proxy stores the malicious copy. There are other tricks an attacker can play as well: https://github.com/golang/go/issues/66653

Ultimately, if you're doing a code audit, you have to compute the checksum of the code that you're looking at, and compare it against the entry in go.sum or the checksum database to make sure you're auditing the right copy.

Related: Proxying can be disabled by setting the environment variable GOPROXY=direct [0]. I put it in my bashrc.

[0] https://www.practical-go-lessons.com/chap-18-go-module-proxi...

The fact that this isn't default is honestly the biggest thing I dislike about Go.

You can do that with Rust as well if you define path of dependency to git repo (or local dir).

Be aware that you cannot publish on crates.io if you do that: either you buy into the system (so that you can ensure that you can rebuild in perpetuity) or not at all (so you end up with a crate that can depenend partially on crates.io, but must always be consumed directly from a repo or directory.

Reply on news.ycombinator.com