Go automatically downloads a newer toolchain if needed
kokada.capivaras.dev
kokada.capivaras.dev
Made a little harder to track down initially as we have a replace directive pointing to a local proto module in each service. It was here that toolchain was set which forced it to keep appearing in the top level module.
Should’ve been opt in behaviour by default not enforced.
We’ve decided to sit on n-1 of Go and I’ve already had an example last month of a well used OSS library setting toolchain to 1.22 as a patch release (which could be considered a breaking change) which fortunately was quickly reverted.
It's not enforced, it's trivial to disable.
go env -w GOTOOLCHAIN=localhttps://go.googlesource.com/proposal/+/master/design/57001-g...
> Many people believe the go line in the go.mod file specifies which Go toolchain to use. This proposal would correct this widely held misunderstanding by making it reality.
That doesn't sound like a good reason to automatically download binaries and run them.
Is it difficult to update or install a new version of Go and are there frequent updates in Go spec introducing new features that it is necessary to auto install the compiler itself ?
Supply chain attacks are on rise and not a new concept and yet we see these changes.
This is not the first time Go lang has introduced a questionable opt-out feature [1]. They backed out but looks like there were no takeaways from that episode.
I'll be raising an issue with my distribution of choice to disable this behavior by default. They're the last line of defense for users' security and privacy, and we as an industry have been trying hard to circumvent them. Maybe precisely because of that reason, who knows.
Go is the easiest language I know to build from source, let alone run a binary distribution.
From what I've seen, the need for frequent updates is more about keeping security scans happy.
But I guess I was hyper-sensible, because I've missed 'rustup'
I should be setting the GOTOCHAIN environment variable to enable this, opt-in not opt-out.
It forces what the foundation wants you to do. It takes away control from you, it's unethical.
Not only that, it opens a can of worms. A compiler should be a compiler.
Not an self updating application, nor an dependency modulator from GitHub. How can I trust it when it does all these things?
Call me old fashioned at 35.
If you want the latest then go download the latest. Is that now to hard for the user?
Just because the latest is out doesn't mean it's any better than the previous version. What happens in a CrowdStrike scenario? What happens when Go gets retired in 50 years?
I don't want to work with the latest. Should I? TCL 9 is getting there but TCL 8.7 is still perfectly operatable. Should I be using 9 because it exists? My work only has 8.6 on production.
So your toolchain updates and they've removed a thing. You've got to hunt down the previous version, let alone needing to discover why it was working yesterday and not today. Unnecessary overhead.
You use a dependency that's not updated for the future version?
What stops someone from crafting a malicious binary? Malware hijacking the download path?
Auto-updating takes away your integrity. Your making blind trust that everything is what it is.
How can I be sure that the updated compiler is the compiler and not a malicious crafted version? If you can't trust the compiler how can you trust your code?
Yes, I could turn it off, but could I turn it on instead.
I shouldn't need to turn it off, I'll update when I want to update tyvm.
lmao, I stopped reading after this.
But strangely I liked this feature, mostly because it allows me to use newer features without needing to wait nixpkgs to update the Go compiler to the version 1.23.
It is easy to disable if you don't like though, something like `environment.sessionVariables.GOTOOLCHAIN = "local"` or `home.sessionVariables.GOTOOLCHAIN = "local"` should do the trick.
You are confusing the `go` toolchain and the binaries generated with Go toolchain.
The `go` toolchain itself is compiled statically, this is why this works without issues even in NixOS. Binaries built with the toolchain may or may not be linked against libc depending if you enable or disable CGO. But this is not related to the toolchain, that will work regardless since the toolchain itself is statically compiled.
Edit: or in other words, the `go` toolchain is built with CGO disabled.
This works even in NixOS, because `go` toolchain is statically compiled.
> My current plan is indeed to remove implicit installations entirely.
[0]: https://github.com/rust-lang/rustup/issues/3635
[1]: https://github.com/rust-lang/rustup/issues/3635#issuecomment...
Here is a slightly contrived, but realistic example of why it is a bad idea:
1) Attacker discovers vulnerability in an older version of the Rust toolchain
2) Attacker creates useful crate and helps it to get widely adopted or becomes trusted contributor to a crate that is already popular
3) Attacker creates and publishes crate changes with exploit code and rust-toolchain.toml to trigger use of older, vulnerable Rust toolchain
4) Unsuspecting developers build the trapped crate or something that depends on it and get owned
Perhaps they could make it a configuration setting that developers could opt-in? That would let developers who want automatic toolchain installs to have it and others who do not want it (or whose employers will not allow it) to not have it.
As for configuration: to me, having it be opt-in negates the entire benefit. My point is that automatically installing the correct toolchain makes it far easier to collaborate with others who aren't nearly as obsessive about Rust as I am.
Possibly, which is why the example is a bit contrived. In most cases, the toolchains will likely be more trusted and be on approved lists whereas binaries created by third-party crates are not.
For more secure environments, explicitness is valued and automatic installation of anything is frowned upon because it can introduce unvetted changes which could include vulnerabilities.
It depends on what work is being done and how much toolchains and ecosystem can be trusted.
If you mean build.rs build scripts, yes, those do run, but it is not arbitrary code. You can view and inspect them before building. If you need more security, you can download all the dependencies and build inside an isolated container.
uhh ya it is. There's also https://github.com/eleijonmarck/do-not-compile-this-code
https://github.com/eleijonmarck/do-not-compile-this-code/blo...
This is true for all third-party libraries. If you blindly download and execute code from the Internet, this is a risk you are assuming.
As I stated above, if you need more security, you can download all the dependencies and build inside an isolated container.
In Go case though the version can only go higher, not lower (e.g.: it will not download a toolchain if the Go version is set to lower than your current one, only higher). So I can't see the same attack being executed here.
That is good to know, assuming that the newer hypothetical toolchain is not vulnerable (e.g. a zero-day in the newer toolchain).
My opinion is still that toolchains (newer or older) should not be implicitly installed without the developer's explicit permission. This could be a configuration setting that the developer has opted-in or a "This package requires toolchain version X. Install it? (y/n)" prompt.
This would still be really difficult to explore:
0. An undiscovered zero-day in the newest version of Go
1. The only way to force users to use the newest toolchain is if your project has dependencies where the attacker has control, so e.g.: they can change their go.mod and set a `go` directive that upgrades to the later version
2. You need to update to this new version, because this will make Go complain that your `go` directive inside your `go.mod` is out-of-date
3. After all this, yes, Go will download the newest version of Go (that is vulnerable)
Again, not saying that this is an impossible scenario, just find it highly unlikely.
> This could be a configuration setting that the developer has opted-in or a "This package requires toolchain version X. Install it? (y/n)" prompt.
I concur at this part though, however keep in mind that this `toolchain` feature is more akin to adding a module dependency, since it reuses the whole module system.
I think there is already lots of trust that is implicit when you are managing modules, and this new feature doesn't make it worse. If anything, considering that the binaries can only come from golang.org/toolchain and I assume Go already check the checksum of every module that it downloads (and also that Go is perfectly reproducible: https://go.dev/blog/rebuild), if anything this is probably more trusted than a random module that you add to your project.
In corporate environment, it might trigger some unexpected alarms (or be blocked)
Gentoo got this right: by default, the "old" behaviour should be kept (no automatic update) and it should only be opt-in
The automatic updating toolchain is not a bad idea, and in most cases it works great. Once in a while though, there's a fuck up and breaks your build or something changes that you didnt expect.
The best way forward is to keep the old tool in a backup location, and have the user be able to revert the update with a single command. Then the user is able to safely return to the old one when the inevitable breakage happens. You could even opt-in to analytics for this action, and the owner of the tool would know if a particular update is a screwup (aka, many people reverted it).
I think you misundertood how this feature works.
If you have a `go.mod` that sets a `go` directive with a version bigger than the version your Go toolchain is currently running, it will download the binary automatically and pass the commands to the new binary. However this doesn't replace your `go` binary in say, PATH, the new toolchain will be downloaded at `GOPATH`, the same as it does with your modules. This is why there is nothing to backup here, nothing is deleted.
Also, keep in mind that if Go is downloading a new toolchain, it means your project will not work with the current toolchain: because if you set `go 1.23` in your `go.mod`, and your current `go` is version 1.22, it would fail regardless until you either fix your `go.mod` or upgrade your `go` toolchain. What Go is doing here is the later, automatically for you.
Only by default, as the Go toolchain supports dynamic linking for ages, and without third party dependencies called via cgo, or stuff like DNS resolution on Linux.
What I meant is that the toolchain itself is linked statically, this is why it can be just downloaded from the internet and it will work without extra setup from the user.
Yes, Go will build the libraries once you call `go build`, and yes, they may link to the system C libraries for things like you said. But the `go` binary itself is static.
If they want magic like this, it should be opt-in, not opt-out.
And yes you need to make sure GOTOOLCHAIN=auto, my colleague had to explicitly set it because it defaults to local on Gentoo.
GoTV builds needed toolchain versions with the Go source code instead of downloading the pre-built toolchain packages.
Maybe you meant "imagine if Python 3.6 automatically updated itself to Python 3.12"... Which would be amazing! Except that Python 3.12 is also not backwards compatible with Python 3.6. There's no saving Python.
All possible places for existing code to break, either via compiler errors, or changes in the expected runtime semantics.
> To ensure backwards compatibility with existing code, the new semantics will only apply in packages contained in modules that declare go 1.22 or later in their go.mod files.
CMake and Rust and Android and lots of other systems have mechanisms like this that allow you to introduce opt-in breaking changes, but unsurprisingly Python doesn't. They just decided it was ok to start breaking backwards compatibility.
Enough of them happened already, that a clarification post was needed,
Not at all. See https://go101.org/blog/2024-03-01-for-loop-semantic-changes-... for reasons.
Please tell me what flag I can pass to Python 3.12 so `distutils` still exists.
I'm not familiar with Python, so I don't understand what is the problem with Python. But I think the problem of Go is unrelated to Python.
Not in fact. Go core team has admitted that Go toolchain 1.22.n versions break backward compatibility: https://github.com/golang/go/issues/66092
There is another breakage case: https://github.com/golang/go/issues/66070 and the team leader doesn't plan to fix it.
This doesn't make sense. The toolchain will be updated only IF you say your project only runs in Go >= XX. This is what `go` directive in `go.mod` says: my projects doesn't work in versions of Go older than this. This is why it happened to me, I declared I need `go 1.23` because of range-over-func, and the toolchain respected my wishes.
This is not an "enforcing you to use the latest Go version, even if you don't wish for it".