Hacker News Reader

Comment by Philip-J-Fry | Hacker News Reader

I think true innovation here would be forgoing a centralised package registry and just using good old file systems. Go modules have the the right idea, pull a package from a server which responds to basic HTTP requests or even a file system. If you want some smart searching capability, or generated docs then create a proxy which packages are downloaded through which allows you to index them.

Just take things back to basics. You shouldn't have to publish a package on some centralised registry, you should just be able to import a package from anywhere.

View on HN

Deno literally did this[1] in order to be browser-compatible[2], and everybody whinged about it. I happen to love that I can just import an HTTPS URL inside Deno code. It's really nice for playing around with libraries, writing little scripts that pull in dependencies, etc.

[1] https://docs.deno.com/runtime/manual/basics/modules/#remote-...

[2] https://html.spec.whatwg.org/multipage/webappapis.html#modul...

I love this approach in Deno! I did write some nice scripts using this!

You can use file:// or git:// versioned dependencies in normal npm `package.json`s today. People just don't, outside some edge cases (local packages, nightly versions) because the centralized registry has upsides for discoverability and maintenance. There's also private registries, where you can setup a .npmrc to pull different namespaces from different registries. But if you want, you can totally be that guy who only publishes their packages to their presumably self-hosted repo - it works.

The same is also true of Python, and once used to be the majority option, which was around the time that the "Python packaging is terrible" impression was starting. Since like 2015, pypi has banned it in hosted packages, but you can do it for private packages all you want.

The fact that Go eventually introduced the central proxy server highlights one of the biggest issues with this approach though: the vast majority of use cases do prefer centralized systems.

The major advantages of centralized systems in my mind are:

* No need to deal with duplicates (what if two indices provide the same version of a package but with different dependencies?)

* Oversight (the ability of most popular package managers to "side-load" from Git repo / URL / etc. "illegitimate" source has been a bane of my existence for a while). Being accepted into some "central" index means at least some (albeit usually not much...) degree of oversight. Being able to load from anywhere means exposure to spoofing.

Also, unfortunately, lots of package management systems in common use can be "subverted" to be used in a decentralized way. And, if it were up to me, I'd rather not have this ability at all than have to try to defend against negative consequences.

Go is exactly how one should not do it, hardcoding SCM URLs on source code, only allowing for source code distribution, and forcing everyone else to jump through hoops to make it work otherwise.

Go programs are a nightmare to package for linux distros due to the way they handle dependencies so I would strongly discourage any other language from copying the Go model.

This says more about Linux distros creaky 90s era design than it says about Go’s model.

Not really. With all the problems of packaging software for Linux, I'll take that any time over Go.

The whole "spirit" of how Go authors approached every infrastructure task they had was to start as simple as possible and grow as necessary, in small increments. Unfortunately, incrementalism doesn't work in this domain. It's very hard / virtually impossible to go back and undo bad decisions once they become public. Some steps will require sweeping changes all across the board and cannot be performed in one increment.

Go packaging mostly works because of the extra restrictions imposed on themselves by most package authors. But, if you were to stray off the beaten path, you'll discover that the system is unprepared to deal with your case. Go is the opposite of thoughtful and insightful design. It wins at first because its fast and easy to do, and once you are hooked, you will have to work extra hard to deal with the difficult parts yourself.

I guess I view Debian et al similarly to how you view Go: distros like Debian have been incrementally iterating on a simple design since the 90s and their package systems have become a Katamari Damacy ball of mud around a core designed for having a single agreed-upon version of each C library shared by all software in the universe.

Now to reliably distribute Linux software, it’s very popular to distribute the software along with the entire userspace the software compiled against in the form of a Docker container.

> it’s very popular to distribute the software along with the entire userspace

OK, there are two ways to interpret the word "popular".

* As in "a presidential candidate enjoyed popular support".

* As in "ski is a popular sport in some European countries".

So, let me tell you this: distributing software with the entire userspace is not popular, if you use the first interpretation of the word. Users hate developers who distribute software like this, this is what they call "bloatware", "not a team-player" etc. I, personally, despise developers who do this, because, from my perspective, these are the low-skill developers who do less to get equal pay at my expense (as a user).

But, of course, it's popular to be a bad developer, in the same way how it's popular to steal bicycles -- low effort, high reward, if you ignore the disappointment of someone else.

Bazel also allows to pull dependencies from different sources via Bzlmod.

1. define dependency like bazel_dep(name = "protobuf", version = "3.19.0") 2. define repositories where to look for it

A repository is just a file structure like /modules/$MODULE/$VERSION + some info about the module. Can be on a HTTP server or just on your local file system.

https://bazel.build/external/overview#bzlmod

I hope more tools adopt something similar. Or maybe a single package manager for everything, so we can finally build cross-language software without having to debug why tool A can't find stuff from B.

Bazel mod requires a registry of packages in a git repo, so it’s quite centralized.

From the documentation: https://bazel.build/external/registry

> Bzlmod discovers dependencies by requesting their information from Bazel registries: databases of Bazel modules. Currently, Bzlmod only supports index registries — local directories or static HTTP servers following a specific format.

It just uses the Bazel Central Registry (BCR) by default. You can specify your own via the --registry flag and then it uses them instead. It is possible to specify multiple registries at the same time, so you can mix the official, company internal and on your local filesystem at the same time.

Reply on news.ycombinator.com