Hacker News Reader

Comment by ryandrake | Hacker News Reader

Ugh. Hate to be Debby Downer. I mean, nice effort, but one of the crappy things about some of these newish languages is how each of them comes with their own package manager baggage that is different from the package manager that all the rest of your software is installed through.

So I have my Debian setup and I'm happily using apt to install all my software. Now ruby comes along and I need some libraries and have to go and install everything with gem. And I can't use apt to see what gems I have installed and vice versa! Argh!!! Now I have to write some python code and here comes pip... And node/npm... and on and on.

apt-get install libx11-dev is just fine for my C and C++ development needs, thankyouverymuch. I don't see the value in yet another incompatible package manager.

View on HN

Once you've started using language package managers, it become very apparent that they have huge value, that apt-get doesn't bring.

In particular, you get the exact version of the library you want. If you've ever had the experience of trying to make your software work with multiple versions of its dependencies because different OSes ship with different versions, then you'll know how annoying this is.

This is particularly true of run-time problems. If they don't compile, then that's relatively easy to fix. If there's a difference at run-time, its often way more expensive to fix.

With a package manager, you get the exact version, but you also can use different versions for different apps on your system. You can also upgrade without breaking other apps (new security version) or not upgrade if there's some good reason not to.

> In particular, you get the exact version of the library you want. If you've ever had the experience of trying to make your software work with multiple versions of its dependencies because different OSes ship with different versions, then you'll know how annoying this is.

Isn't this just kicking the can down the road though? The incompatibilities are still something you either have to deal with or users end up with 58 outdated versions of each library and the security vulnerabilities they bring.

> Once you've started using language package managers, it become very apparent that they have huge value, that apt-get doesn't bring.

This is what's wrong with so much modern development, we do what's easy for developers, not what's best for users.

> This is what's wrong with so much modern development, we do what's easy for developers, not what's best for users.

In the client software case, software that works is what's best for users. If you upgrade a library and apps start working, that sucks.

If you're running server-side software, you want dev-production parity, or you can't know it works. So you _need_ to use a package manager. But the same applies to client software: you can't test all these permutations, so you can't even tell it works, and it probably doesn't.

You don't think software that's secure is important for both cases? Open up a random project with a language dependent package repository and tell me how many of the packages are up to date? More often than not the answer is zero with commercial software.

If you're running server side you should have test environments that match prod to be able to catch incompatibilities early.

I'm not arguing against using a package manager. I'm arguing against using 5 of them that don't know about each other.

I am interested in such a thing (e.g. ruby bundler knowing how to install native dependencies from any package manager), but that means pkg-config or similar ubiquitous tool would have to be able to query all kinds of servers in order to see the versions of packages available.

FreeBSD has had a package called bsdpan for some time that will register items installed through CPAN with the package manager. You may want to poke at that for inspiration on how to do it for other languages. Offhand, for handling multiple systems, it may be most useful to check something like uname and parse accordingly to figure out what's what.

For what it's worth I completely agree with you. I've recently started learning Rust and the only thing that truly bothers me about it is that it appears as though you _must_ use Cargo to manage your dependencies.

I work for a company that supports a distribution (SUSE) and I also participate in a distribution's community (openSUSE) and this mentality of "let's just reinvent package management" really does bother me, because it feels like the benefits of having a single package manager for the entire system is being ignored. And this ease of pushing new packages results in a huge amount of small packages so that a standard "hello world" in many frameworks requires something like 700 packages (from a distribution side, that's 700 downstream packages that we now need to curate and maintain).

Don't get me wrong, language package managers do have benefits, but they make life absolute hell for distributions. And distributions having full control of package management definitely brings benefits to users. Rust is the most extreme example, where projects can require a particular version of the nightly build of the compiler. How the hell is a distribution meant to be able to build and ship Rust code in that instance (given that distributions also provide the development tools necessary for working on said projects)? Do we create a new package for every nightly compiler build? How do we curate it?

One of my colleagues gave quite a fair review of the situation and some potential future steps in his talk "Package Managers all the way down"[1]. But bringing this mentality verbatim to C++ is not a step in the right direction IMO.

[1]: https://www.youtube.com/watch?v=4ua5aeKKDzU

You don't have to, it's just the path of least resistance.

We've been working with distros on packaging stuff; it's mostly sorted. Each distro is doing the thing that works for them.

Additionally, making sure we play well with other build systems is a big goal this year generally.

Also, for cargo's usecase (mostly-reproducable builds, a space that rubygems also attempts to fill) system package managers are simply insufficient. Nix is the only one that comes close, and you

A) can't remotely expect anyone to have that installed already B) have no control over the repository or toolchain

> We've been working with distros on packaging stuff; it's mostly sorted. Each distro is doing the thing that works for them.

Have you worked with openSUSE, since as far as I know devel:languages:rust is still in the state of "we aren't sure how to deal with these problems". In particular, there aren't any rust rpm macros in openSUSE yet, so there doesn't appear to be a clear way to create an rpm for a particular Rust project.

I've written several things in Rust that I would love to package for our distribution, but I just don't know how we are meant to do it without depending on a network connection to download dependencies out-of-band (outside of rpms dependency system).

> Additionally, making sure we play well with other build systems is a big goal this year generally.

Yeah, that's the big problem at the moment. OBS provides huge benefits (rebuilds when dependencies change, tracking of security bugs and fixes) but it also has reasonable restrictions like not being able to connect to the internet.

Having a better story in this respect _really would_ help Rust in its adoption in distributions.

OpenSuse is a bit behind, say, Debian, but I thought there were people actively working on it. Las I remember, Rust and Cargo were packaged, but they weren't sure if they wanted to follow in Debian's footsteps and automatically convert packages or do something more manual like Arch.

As long as ou have the stuff previously downloaded somewhere, offline builds of everything are totally supported; they're required not only by distros but by Firefox, for example.

Let's say that you and I both create a library. You try to publish your C++ library to the standard debian repository, and I'll try to publish mine to cpam, npm, pip, etc...

If you actually manage to get your library included, then we can compare how long it takes to update your library.

Sure. I publish to deb repos all the time.

(I've found Bintray to be the easiest.)

I didn't say "a" deb repo, I said "the standard debian repository". Just like when I'm talking about NPM, I'm talking about the main NPM repository, not my company's private NPM server.

Well, you've chosen a terrible comparison then.

How many NPM repos are there? Just one, too-big-to-fail, centralized repository.

Don't ding apt/deb just because people setup their own repos rather than put all the code in the world into one bucket.

There's a lot more inconvenient things than adding the /etc/apt/sources

> How many NPM repos are there? Just one, too-big-to-fail, centralized repository.

There are actually plenty, it's just that they're usually found on-prem at a company.

Systems like Debian, though, have the ability to add custom repositories. So after the initial, possibly small package that just configures the sources.list, everything's handled with all the other updates. That's something that npm, etc, can't do.

Npm and many other package managers support custom repositories just fine

The reality is that these tools add value that extends beyond mere package management. You are right though, if you just use them to install things, and not as part of a development process, they might seem superfluous.

Well I think exactly the opposite. In C++ it's a nightmare to build a prototype with a new library. You spend a huge time setting up the build system for that particular library, need to figure out how to plug into your code, fix dependencies (which in practice always have something broke/missing). That being, I think a propper package manager would add value to c++ dev,

> apt-get install libx11-dev is just fine for my C and C++ development needs, thankyouverymuch

... on Linux.

Common problems with using distribution package managers for software development:

- not all versions are available, and the versions which are available is different across distributions

- even if the version is the same, the location is not consistent, necessitating search logic to find libraries to link to and headers to include

- even if the version is supposedly the same, the distribution may have enabled different options, fiddled with the package or otherwise made it unsuitable for your use. Upstream developers often loathe to support the version your distribution uses.

- multiple versions cannot be installed independently, and the version you need may conflict with the packages you are using on your system but are not dependencies of your project

- This likewise can cause conflicts between different projects you are working on, causing a need for VMs and projects like vagrant, which are very heavyweight and mean you need to maintain multiple copies of your preferred editor and IDE setup.

- Forking a dependency to fix or customize it (to submit a fix upstream or not) gets even more painful.

All of these are much less painful for language based package managers.

In theory there is no reason why build tools can't be integrated with package managers so you don't experience that kind of silliness.

For development purposes, you may want to have the system build everything from source (and/or fetching pre-compiled modules from a build system repository), and output a self-contained tree of local files that you can run.

For release purposes, you can in theory output a package manager package (RPM, .deb, etc.) that contains only your own code's binaries and dependency declarations that the package manager can go satisfy.

There seems to be some degree of integration with build2 (see https://build2.org/faq.xhtml#why-syspkg ) though I don't know how much exactly or whether they go as far as what I'm describing.

> apt-get install libx11-dev is just fine for my C and C++

Language level dependency managers tend to cover a much wider selection of the extant third party library ecosystem than system level package managers.

Because Debian isn't the only game in town for OSes?

I for one don't want to bother with M×N packages.

Reply on news.ycombinator.com