Hacker News Reader

Comment by digi_owl | Hacker News Reader

As more of an admin kind of guy, i find this explosion of language specific repositories worrying.

I have already had to deal with python code that do little to no dependency documentation, that just carpet bomb my system with stuff pulled from the net at the install stage.

View on HN

Python has/had a terrible packaging story though. As you said, dependencies weren't really documented. Rust, on the other hand, documents all dependencies in Cargo.toml. And dependencies can easily be parsed from Cargo.toml without the need to execute any code (as would be required in the case of python)

> Python has/had a terrible packaging story though. As you said, dependencies weren't really documented.

I wonder where have you lived? I rarely found (if ever) an undocumented dependency list for any serious Python project in the last couple of years.

Are you talking about requirements.txt? Requirements.txt falls short of cargo or really any other mainstream language's dependency doc (no reproducible builds, for one thing -- effectively almost as bad as undocumented).

Requirements.txt + virtualenv, everything is fine.

Really, significantly easier to not screw up in a large org than bundler, because dependencies don't leak by default.

Actually, `bundle exec` is way easier to use than virtualenv.

Mostly requirements.txt, yes. It's a standard and it has been a really long time I used a package where I didn't find one.

What you mean by no reproducible builds?

python's packaging, especially compared to state of the art which cargo represents, is really, really poor. documenting direct dependencies is just a start. requirements.txt is somehow simplistic and complex at the same time (remember about the special options which thankfully aren't used too much) and you can also list dependencies in setup.py, too.

there's the Pipfile initiative which gives me much hope, though :)

(I hope I'm not misunderstanding your concern), but a quick search reveals that Rust prefers to statically link, so in this case, remacs would distribute as a 'single' binary. Only the developers (of remacs) would need all the dependencies/crates.

This creates its own headaches as a fix for the next Heartbleed becomes not "upgrade this one shared library", but "upgrade these dozens of individual programs when an update becomes available".

There has always been war b/t static vs dynamic linking. The debate will continue forever, but it doesn't really matter anymore. Each side understands and has solutions to its deficiencies.

Fwiw, lisps have a somewhat interesting third way: you distribute everything in a single image like a statically linked program. But, since that image includes the compiler, it can upgrade itself.

Or they include a loader and can load code, code which may be compiled somewhere else.

Programs all need to update all the time anyway.

language specific repositories worrying

What is the alternative? Every major Linux distro has its own repository, as do all the BSDs. The picture on Windows and macOS is even more convoluted. Are developers in all these languages supposed to support the packaging of their software in half a dozen different repositories?

Maybe all programmers should get together and build "one repository to rule them all"? I have no idea how we herd that many cats!

Sounds like xkcd 927 ...

FPM is at least a step in the right direction to reduce the overhead of creating all of these.

At a minimum, Linux should have a common packaging system; though there may never be agreement. It's hard to see macOS and windows endingnup with a common one.

But maybe we could do a third party option that is independent of each OS? Those rarely work out well though.

And that is forgetting that are many more OSes out there than just those three.

Of course there are many more, but if you could build the one true package manager for all the major OSes, my guess is that the smaller ones would join the fray.

Not really, because it would meant that their semantics weren't accounted for.

As an example, IBM just released yesterday a new mainframe model.

z/OS is a totally different animal compared with POSIX based OSes or Windows, starting by the file system, catalog based.

Nix(OS) is basically that... If only we could get upstream to start building with Nix

Ugh...no. I tried Nix until I found that all packages that depended on OpenSSL were broken on OS X. I checked for a bug report and found that one had been filed and sat open and unfixed for 18 months. Given that this bug makes Nix basically unusable on OS X, it seems like developers of Nix are uninterested in becoming anything resembling a universal package manager.

Meanwhile, cargo works flawlessly on every system I've tried (Linux, Mac and Windows). Sure, it only compiles Rust programs, but there's no way that I'd advocate moving those programs to a package manager with as little regard for such a large part of its potential userbase.

> Given that this bug makes Nix basically unusable on OS X, it seems like developers of Nix are uninterested in becoming anything resembling a universal package manager.

Or that they just don't have enough manpower to actually support all the platforms.

with python the rule should to be to run every deployment at least in its own venv.

That's not so practical if the application is a text editor, or some other administration tool (debugging, statistics etc).

My rule of thumb is to never install anything into system Python by pip, etc. If it's not an OS package, it belongs to a virtualenv.

I more or less have this rule, except I generally will use the --user flag to install things on a per-user basis if it's not being installed for a particular project.

I recently came across this: https://github.com/spotify/dh-virtualenv

For a system utility, you can still use virtualenvs. The launcher script just needs to use a shebang like:

    #!/usr/share/spiffyapp/venv/bin/python

It really does surprise me that languages come with their own package managers. In most linux contexts, there's really no need for it since linux distributions come with package managers.

Part of me feels like a better solution is to maintain older branches to give distributions opportunity to catch up.

> In most linux contexts, there's really no need for it since linux distributions come with package managers

This single sentences hints at least major two reasons for language-specific package managers:

- non-Linux platforms

- there's many different Linux package managers (apt, rpm, pacman, nix, portage, ...)

A language-specific package manager means developers (i.e. the ones writing the packages) only have do to packaging work once, instead of for every platform they might want to support, including ones that they have no way to test on (and a lot of code, especially in the modern wave of languages like Rust, often works on all platforms with no particular effort from the developer).

From an asymptotic perspective, using the platform package managers results in something like O(# libraries * # platforms) work and likely a lot of code not being packaged for some platforms, whereas language-specific package managers is something closer to O(# languages + # libraries) work (or maybe O(# languages * # platforms + # libraries). Looking at this decision purely in terms of developer effort, the latter seems like the better choice.

(There's also things like fpm that are designed to be more like O(# platforms + # libraries) work, which is nicer than either of the above!)

For anything but C, one can't really trust the distributions to care enough for that to be an option. You can install Common Lisp libraries from the Debian repositories, but if you do, you'll find yourself in a world of problems since they are mostly severely out of date, if they work at all.

Far too many times have I had to explain to newcomers to Common Lisp that they can't install stuff from the repositories, and instead have to download their CL implementation separately and then use Quicklisp.

Reply on news.ycombinator.com