Go modules and the domain expiry problem
utcc.utoronto.ca
utcc.utoronto.ca
> It is IMHO the strongest security guarantee you can offer without forcing each module author to manage keys.
Outsourcing the problem to DNS still forces module authors to manage keys. Not keys to sign code, but keys to sign TLS certs nonetheless. Granted, authors can further delegate managing TLS keys to other people, but you cannot escape the ultimate issue of managing control of the domain (passwords at registries at least).
If we cannot get rid of key management anyway, can we split the problem into 1) Code Signing and 2) Code Distribution? Once we get the Code Signing done right, Code Distribution will be much easier with robust solutions (e.g. content addressable storage and various kinds of proxies/caches for high availability).
Now code signing, lots of schools of thoughts… On the extreme end of decentralization, PGP/GPG people will say Web of Trust and Crypto/web3 people will say hardware wallets; while on the other end of centralization we seem to be getting by just fine with publishing public keys on GitHub and various social networks. Just pick a place where you Feel Good Enough™ :p
For Go where you're just publishing sources with a build necessarily, you could opt for SigStore git signing so you can at least validate the author via a 2FA auth provider's OIDC token.
Using a personal domain name should probably be discouraged unless you’re sure you know what you’re doing, and even organizations might want to think hard about whether they’re really want to maintain a website indefinitely.
Unless Github or Gitlab goes out of business. Github might not be much of an issue, but what is some shifty company buys Gitlab during the next financial crisis?
Github has Microsoft money, so I wouldn't be to concerned, but even yanking Gitlab could break an insane amount of modules in one go. Having modules distributed among many domains reduce the chances of a large scale issue, for the cost of many small problems.
Not saying this wouldn’t be disruptive. But it’s not a project ending scenario.
https://www.digitalocean.com/community/tutorials/importing-p...
>import "github.com/google/uuid"
This should be just "google/uuid".
"google/uuid" tells you nothing about the owner. It might be Google, or it might be me, who squatted the name early on.
"github.com/google/uuid" would be much difficult to squat. Also, it tells you that this comes from the Google user in GitHub, which is something much easier to verify.
"code.google.com/go/UUID" would be even more of a guarantee. But I guess they decided relying on GitHub to host their Go packages was good enough.
Plus if you really don’t want a full domain in your projects import paths then you can vendor in those modules, albeit at a cost of higher maintenance for yourself.
The biggest problem with having the domain name included in the import path is just purely visual noise. It’s pretty damn ugly. But that problem is purely aesthetic.
I will grant you that when package versions were encoded in the import path, that was extremely problematic (to say the least). But that’s gone away now we have proper module versioning (and I say that even as someone who doesn’t particularly like the solution that Go finished up with).
> Third-party packages are always known by their fully-qualified names, which include the site that hosts the code (e.g. github.com), the user or organization that develops it (e.g. google), and the base name (e.g. uuid).
However this isn’t a uniquely Go problem. You have the same problem when switching to a new module in any language and the reasons you’d struggle is the same: because the existing module has been abandoned. If it hadn’t been abandoned then it’s very easy to find the new git repo and thus update your Go project.
> git decentralization has nothing to do with it, unfortunately.
It has everything to do with it. You have a local copy of the repo on your development machine and thus can throw a new mirror up if you need to. All you lose is any issue tracking and existing maintainers. But if you’re having to resort to this, those maintainers have likely already abandoned that module already.
This way I can have a backup of the dependencies wherever I want, even when not using "go mod vendor" (which I like also a lot tbh).
So I don't understand the critic in the article, you cannot solve this problem efficiently other than putting everything on a blockchain, which will waste a lot of hard drive space.
Apart from the obvious bandwidth and storage space wasted, this is also a problem when compiling an SBOM (Software Bill of Materials) and when scanning for vulnerable dependencies. There is simply no way to tell if your Go binary is vulnerable simply by looking at the go.mod/go.sum file, you have to build and analyze the binary and hope that everything went well. Same for compiling NOTICE files for license compliance. (wink wink Apache license)
This isn't an unknown or unsolved problem either, Maven solved this ages ago.
There are also some minor nitpicks, such as their absolute requirement to do path-based semantic versioning from v2, but not from v1 or v0. If you change anything, but absolutely anything that's publicly visible, you must increase the minor version of the library you are publishing or you will have a really bad time. If you want to publish both a library and an application, you have to either version them with ridiculous version numbers, or you have to split the two.
This is a problem because every time C has a vulnerability, you get a deluge of vulnerability scanner alerts or Dependabot pull requests to update it, even though your application has nothing to do with C.
If anyone has a solution for this, I'd really love to hear it because it consumes a silly amount of time to deal with bogus vulnerability alerts.
> (In Rust) there will be namespaces soon anyway, just not the kind that would require a huge paid manual review/support staff to administer everything
Is he talking about Maven Central?
Because using DNS prefixed packages as a means to avoid namesquatting seems to me like the best solution out there. For all its faults, Maven got a lot of package-management things right, and I'd argue DNS prefixes is one of them.
I wish that solution wasn't handwaved so quickly.
You just make an account at Maven Central, prove ownership of your domain name, and can then start publishing packages with your publisher credentials. If you suddenly lose your DNS name (as in the case of .ga), you'll still be able to publish packages for a while, at least until there is a new ownership verification, giving you ample time to publish new versions with deprecation warnings.
But, of course, that requires people on top of the registration system, approving requests, attending issues. Claiming that the need for such staff is something to avoid, sounds to me akin to if we were in the 90s thinking how to distribute software packages in our Linux distro, and someone said the need for package maintainers would be something to avoid.
I guess Go sidesteps this by not having a registry, and Rust seems to require you to explicitly say which registry a package comes from. But pip is vulnerable to this issue because it still doesn't have namespaces.
Luckily the go ecosystem has a number of tools for dealing with this. You can set up a local trusted proxy of all the code you use internally and point your builds to that instead of Google’s default proxy or just doing direct pulls. Or you can vendor the code you’re using into your own repository. You can also easily alias specific libraries to point elsewhere if specific ones are problematic, without having to change the code directly.
If control of crates.io or npmjs.org change, how many rust/js projects will still build?
The trade-off seems to be between distributed vs centralized. Where in the distributed case, you take on more risk that something will break for a lower chance that everything will break. And in the centralized case you have a lower chance that something will break, but if anything does, you're completely SoL.
Now, I think there's a conversation to be had on that trade-off, but it doesn't feel like the article really covers it. Centralization has some really nice day to day benefits, but it's not exactly a trade-off with zero downside.
If my package is at example.org/pkg/mypkg and I stop paying for my domain name for any number of reasons then that package is essentially lost.
By default go mod fetch packages from Google proxy/cache.
Doesn't sound like a good situation to me.
With the bonus that if github does go down, you can redirect it to another mirror of the package if the maintainer puts one up (or you can vendor early, but that's an option for everything).
It seems you have two specific options.
Centralize on a big company's url, which you can use in go via sticking to github.com (and a couple other domains) packages as well as js/rust/java/etc.
Decentralize it and have to worry about constant low-level package rot, but not be able to have any individual domain take out all of your dependencies, which you can only really do in go (or languages like C where git submodules are a fairly standard form of dependency management).
There is a tradeoff of having a single point of failure in the first example in exchange for increased reliability during normal operations, and in the second example, you're much more likely to be hit by package rot, but you're significantly less vulnerable to your entire dependency tree being wiped out. This is the tradeoff I'm talking about.
https://proxy.golang.org does not save all modules forever. See FAQ on the site.
For a long time I did not even realize that all of my Go package installations were going through Google-controlled infrastructure by default; something I'd object to had I known sooner. The cache also tends to serve the last working release, rather than the one that's actually the latest, so you sometimes don't even realize that the tip is broken. I can forgive most things, but not hiding errors, especially as I bash my head against the keyboard. On top of that, proxy.golang.org has historically not been playing nice with upstream repository servers[1]. I tend to keep GOPROXY=direct in my environment for these reasons.
In particular, I don't see any reason why the package-name needs to be related to the mailing-list/bugtracker, or to the location of the canonical code repository.
XML uses URLs as labels for namespaces; they are just labels. Nobody expects to find a document about the namespace, nor a schema, at the namespace URL. There's never a need to resolve a namespace URL. Why should it be necessary to resolve a Go package URL?
[Edit] I honestly don't know, because I've never used in Go.
That's in itself a really simple system and doesn't rely on a single third party, but obviously comes with the ownership issue (unless when pinning a specific commit) - in my opinion a good compromise.
Your argument just favors distorting the concept of URL to use URL as URN. But if we were going to use URLs to identify abstract resources, why don't we use URNs instead?
Not quite.
In go, you import 3rd party packages via a URL where a VCS can find them, eg.:
import "github.com/gorilla/websocket"
Someone famous once said "Good URLs don't change". Turns out he was pissing in the wind; nobody paid attention. And the market in DNS names guaranteed he'd be wrong.