Hacker News Reader

Comment by throwaway894345 | Hacker News Reader

Yeah. Not sure about BSD, but I was wading into building the GNU coreutils and other GNU packages just yesterday. Fresh hell, they all seem to be build dependencies of each other. ‘sed’ is its own build dependency.

The whole C ecosystem is a joke with respect to builds—all dependencies are implicit; you’re just expected to have the exact dependencies installed on your system at the exact versions and in the exact locations on the filesystem that the build tooling is willing to look. Autotools and CMake are absolutely terrible; they make Python’s build ecosystem look sane and well-designed.

So no, “security” isn’t the most compelling use case (for me, anyway), it’s moving past these dumpster fire build systems as quickly as possible so mere mortals can build their own software. Specifically, hasten the day where my entire application dependency tree doesn’t bottom out in some Autotools, CMake, shell script, project.

View on HN

> Yeah. Not sure about BSD, but I was wading into building the GNU coreutils and other GNU packages just yesterday. Fresh hell, they all seem to be build dependencies of each other. ‘sed’ is its own build dependency.

This is an area where the BSDs are pretty good. There may not be a dependency system here exactly, but because XBSD base builds XBSD base, everything you need is in one checkout. Generally with just (BSD) Make driving the build.

Programs outside of base are genrerally built with Portfiles which specify dependencies such that (BSD) Make can satisfy them. But somebody has to write the Portfile, and it's often not the authors of the program.

Agreed on a hundred thousand.

I used to feel sheepish defending npm and the whole node package ecosystem against its critics, but once I started trying to run deep learning applications or WebRTC media servers I quickly realized that some of the critics are probably coming from a much worse package management system that they’ve merely grown familiar with.

Want to have two pieces of software on one computer that rely on... idk, two different versions of libwebsockets? Yeah, you’ll need either docker/overlayfs, or some filesystem or pkgconfig hack that I’m sure a comment reply will helpfully mention if you want two have to versions of a library installed for two different executables.

"two different versions of libwebsockets"

If your programs depend on different MAJOR versions of libwebsockets and libwebsockets' build system doesn't easily allow that to happen then complain to the developer of libwebsockets about it. If the developer of libwebsockets is not interested in solving this problem then complain to the developers of the programs which depend on libwebsockets that they should find a better library which doesn't suck. If they don't listen, rewrite the tools you depend on to not suck.

If, on the other hand, the programs depend on different minor versions of libwebsockets and libwebsockets broke API in such a way that just using the highest common version doesn't work then complain to the developer of libwebsockets that they should stop doing terrible things like breaking API and then complain to the developers of those applications that they shouldn't let this kind of breakage slide and should move to a different library.

If you think this is unrealistic then I think your push towards insane let's-pull-all-the-dependencies-into-one-binary/package build systems is unrealistic.

You want a sane build system? The change starts when you stop tolerating idiots who cause things not to work.

We don't tolerate engineers who build crappy bridges which break and kill people. You shouldn't be tolerating programmers who build crappy libraries which break API or don't allow two major-versions to be easily simultaneously installed and linked against.

My issue is that the default behavior is to install these libraries globally, even when they could probably be installed locally for the thing you’re trying to build. (I picked libwebsockets out of the air, I don’t necessarily have a complaint with that particular library)

On my old MacBook Pro I have dozens of node projects that I’ve downloaded over the years just to mess around. If I ever want to clean up, I can delete a project’s folder, and know that everything it installed is now gone (because it would be in its node_modules folder).

On my linux desktop if I want to mess around with a cool project that’s written in C/C++ I immediately pull out Docker and start trying to build up an image, since I don’t want to pollute my global library folders with libraries that I might be downloading just to try out this one thing. I can then delete the docker image if I want to clean up. This works, but it feels like a workaround to the main problem of libraries being global.

You seem to be confused about how typical C/C++ projects are built. If you need globally installed dependencies, you're going to have to install them yourself. A typical makefile is not going to download a long list of random junk from the internet and install it your machine without asking. Although you'd be forgiven for thinking so if you're used to npm.

If you don't like where the project installs its binaries or libraries, you can configure that before `make` or `make install`. You can install it to your home directory, have a separate directory tree for experiments, or whatever you want, and clean it up whenever it is convenient. You can also install things in usr/local and just ignore them. The system is actually well designed if you learn how to use it.

If you need a dependency, use your package manager to install it and you can trivially delete it later... and if you try five different frontends for the same library, you only install one copy of it.

Docker is a workaround for a lack of Unix awareness and sysadmin experience.

The system is not well designed. For example: if I have 3 projects I'm developing locally, each needing a dozen (somewhat overlapping) libxxx-dev dependencies to build, how can I automatically clean up the dependencies that are no longer needed when I delete one of the 3 projects?

Distro package managers are focused on providing a good experience for sysadmins while ignoring needs during development time. Unfortunately, docker and the developer-centric improvements in language package managers have come at the expense of worse sysadmin experience.

The situation will remain sub-optimal until both groups start working together to address both needs.

If you've got some leftover libraries in /usr/local/lib on your development machine because of an experiment, why is that the end of the world? If you're worried about it, you shouldn't be.

The worst thing about Docker is that people think learning Docker is a substitute for basic competence as a sysadmin.

I'll put it in context.

Let's say I have leftover garbage in usr/local/lib. I clone a new project that I need to patch because it's broken. I try to fix the new project but the fix isn't working. I have no idea why. I spend hours trying to trace down the problem.

What was the issue? The project picked up some of the old garbage libraries I had in usr/local/lib, which had a bug that made my fix not work.

For developers, global dev library garbage = global mutable state. It interferes with development, creates more sources of potential issues, and takes time to trace and debug. A clean and/or localized environment is essential for the accurate pinpointing of bugs and issues.

> I spend hours trying to trace down the problem.

This is the problem. Once you understand library loading, once on your system, and learn, once, how to query the system and see what libraries are being loaded and from where, you won't have to waste hours on this problem again. Until that point, you're operating a system without understanding, and that will always lead to breakage.

I've installed hundreds of pieces of software from source, and I have a fairly decent idea of how linking and loading works. I still have to spend time figuring out why something doesn't build when it should due to something strange about my system.

Of course I understand how it works. I just can't constantly keep in mind what garbage I played around with 3 months ago that might be responsible for the issue I'm looking at right now. I don't expect it, and more importantly, I shouldn't have to expect it.

It's a basic tenant of development. Eliminate all additional sources of issues and complexity by presenting a clean, well-controlled sandbox in order to effectively diagnose it and fix the real bug.

In fact, you should be able to try a newer version of a dependency, an older one, or run 10 variants (containers) each with different versions to e.g. test whether the project you're working on is widely compatible with a flexible range of that dependency. These things are done all the time.

The tools are inadequate, not the developers. Engineers on both sides need to stop assuming everyone else is an idiot.

As someone who’s spent a lot of time as a sysadmin over the years, your last comment sounds backwards to me. The DevOps trend has pushed more general awareness and Docker has played a key role in three ways: making explicit automation requirements for the setup process (Dockerfiles have to work, unlike that MS Word document someone last updated 3 years ago), being private by default and requiring someone to reverse engineer the system to learn which things should be shared, and being immutable means that teams are forced to document shared state and backups rather than leaving messes behind for ops to deal with.

The key to understanding the benefit is the power of having done this in a standard, widely used way which is supported by many tools, used by an increasing fraction of projects, and which makes knowledge and training broadly portable. Some shops had some of those benefits but most fell short in multiple areas, ran into arguments with vendors / contractors, or required substantial ramp-up time for anyone new. Distribution packaging solves some problems very well - there’s a reason almost everyone uses it inside containers - but it has well-known problems with conflict resolution and coordination which are not easy to solve, tending to lead to inflexible policies and restrictions. Having a standard answer for the boundaries between generic system-level and application-specific concerns has been a great way to improve both sides while saving a good deal of time.

> The key to understanding the benefit is the power of having done this in a standard, widely used way which is supported by many tools, used by an increasing fraction of projects, and which makes knowledge and training broadly portable.

You're talking about Makefiles here circa 25 years ago, right?

Makefiles have to work, and /bin/sh is still there last I checked on my machine. The problem with the broadly portable knowledge and training is that many people seem to think "don't bother with all that, just use docker" is an acceptable answer and "best practice" is not to learn Unix because it's not new and shiny.

Saying makefiles solve that is like saying screws solved construction. They partially address only one of the concerns I mentioned, doing nothing for the others, and as the numerous tools in that space (automake, cmake, etc.) show they’re hardly a complete solution even for just the challenge of building things.

> Makefiles have to work

And yet I get GNU Makefiles all the time that don't work when I type "make" at my computer, since they use things that aren't compatible with my version of Make. It's clearly more of a problem than you think it is.

The benefit of global libraries is that they are easier to update or patch. If there is a vulnerability in an npm package, you will need to update it in all your node projects (and/or rebuild all your docker images analogously).

A hybrid solution where the package manager is aware of all installed versions, can symlink to a globally installed copy and can update all symlinks would probably work.

Unfortunately, there is very little interest in fixing the problem on both sides (distro PMs and language PMs) since they both believe their own approach is right and don't care about the other group's problems.

yarn uses a global cache and symlink if I remember correctly.

and npm has an "audit" feature, which is default on, so there's quite a bit of movement in JS land to address the disadvantages of having a myriad small packages.

Its not about the size of the packages, its about managing an entire system, for whatever definition of the word "system"

In the "olden" days, you would patch the vulnerable dependency and be sure that all services running on the machine have been updated to the new version. (in theory - if everything works great and there is ABI compatibility etc etc).

Since the library is installed in a well-known system location, there are only a few paths in the filesystem that you would need to check to make sure it's up to date. This makes machine maintenance easier.

Nowadays, to patch a vulnerable dependency you need to potentially go to all individual projects, check whether they depend on it, update the version for that project, make sure to rebuild any containers that... contain it. Rinse, repeat.

There is also not a single place where the dependencies live, so you can't just run the same command across multiple machines to do the check - you have to check the directory structure of every individual service.

This can be very slow and painful.

Agreed. Yet it seems the ecosystem simply accepts this trade-off. For increased productivity, increased memory-management security, etc.

My experience has been that, for sane libraries, the configuration process supports installing to an arbitrary prefix. In general, I end up making a new prefix for every project. And if doing this turns into dependecy hell, shouldn't that make you think twice about your dependencies? Worst case, you find out you need a different libc, and then you start questioning your life decisions.

It sounds like you just want a good package manager.

On my system I have built lots of random projects to "mess around" and to achieve the same effect of cleaning them up I can just make a list of all the packages I explicitly installed to make them work (including packages which I already had) in a simple flat file. I can then feed that file into my package manager to delete only packages which aren't used by anything else.

Now, I can hear you say: "this isn't practical, these projects depends on 400 dependencies". The question I would then ask is why has it become commonplace for a "dependency" to sometimes equate to 20 lines of code. For these "dependencies" sure I think maybe some kind of local storage is appropriate, but I think that this is already solved in languages like C using git submodules. I don't personally like this approach myself, I would rather people just flesh out a standard library or other common libraries than to produce a 20 line snippet that everyone includes in their project through submodules. But the fact of the matter is, this is still an option.

The other thing I should point out is that your docker image is surely going to take up an order of magnitude more space than just installing a few libraries and keeping them around. This in turn means that you could have just kept an order of magnitude more projects in the same amount of space. And finally, the approach of putting everything in one directory is a big hit in terms of disk space. I don't know much about node or JS but I have to assume that the language is still primarily text based. Keeping around hundreds of dependencies written in text (even with minification but does that happen with node?) is going to take up a lot more space than a few shared objects. So in an ecosystem where everyone has decided that disk space isn't so important, what is the point of easy cleanup?

Given the wording you use by "pollute" I assume you literally mean having those files there in the first place. But I don't understand what the problem would be with that. Especially when most projects use dependencies you're most likely already going to have installed. As long as you don't work against your package manager, nothing should ever break as a result of having too many packages installed.

> Want to have two pieces of software on one computer that rely on... idk, two different versions of libwebsockets? Yeah, you’ll need either docker/overlayfs, or some filesystem or pkgconfig hack that I’m sure a comment reply will helpfully mention if you want two have to versions of a library installed for two different executables.

Just setup the correct flag / env variable. That's not named a "hack", that's named knowing how to use a computer.

If you do not know how to compile and link an hello world in C. Then indeed, you should probably stay with an hello world in NPM/JS and its 1500~ packages dependencies.

There are a lot of package ecosystems, but more packages, and one of these packages will always depend on something implicit.

Building a new system is not going to help. Xkcd 927 about standards applies

I agree that we shouldn’t try and wrangle npm, gems, cargo, etc. into one package manager. I have no problem with different communities having different package ecosystems.

My main problem is that when it comes to C/C++ dependencies, the package ecosystem defaults to installing globally and has no standard way to specify dependencies aside from having instructions to “sudo apt install these 20 libs”.

> Want to have two pieces of software on one computer that rely on... idk, two different versions of libwebsockets?

You've pointed this out yourself, but this works just fine in any sensible OS. You just have two or more versioned instances of libwebsockets in your /usr/lib directory, and a symlink (or multiple symlinks with different partial-semvers) that points to some appropriate default. I think even Windows is doing something like this nowadays, as part of the whole WinSxS mess.

Why does a "default" need to exist at all at the system level? Why not just have various versions available and have version decisions be local to the software being developed/used.

Security. By having defaults the operative system can share the most secure minor version of that library (or multiple major if necessary). When you have a copy of a library for each program then most of your programs will end up using obsolete or unsecure dependencies.

Plus it also cuts down on disk space usage.

It doesn't. Just don't install it if you don't want it.

It doesn't, strictly speaking, and in fact, it barely does.

It's usually just a symlink from libwhatever.latest.version.number to libwhatever.so, on the theory that you probably want the latest version of the library (with all its improvements and bugfixes).

There's sometimes an intermediate symlink to the newest major versions too (libwhatever.latest.so, libwhatever.2.so), to avoid breaking changes.

> Fresh hell, they all seem to be build dependencies of each other. ‘sed’ is its own build dependency.

Except that people have run into the exact same issue with stuff like rustc and cargo. Bootstrapping an entirely new platform is just hard, no way around it.

Comparing coreutils to a self-hosting compiler is a strained analogy. Bootstrapping a compiler is about as hard as it gets; there’s no reason coreutils needs to depend on itself.

There's "no reason" why the rustc build should depend on cargo, but apparently it does. And this makes it way harder than it should be to keep rustc support up-to-date in any distro, which in turn impacts stuff like Firefox updates.

> There's "no reason" why the rustc build should depend on cargo,

rustc is a Rust project. Cargo is a build system for Rust projects.

> this makes it way harder than it should be to keep rustc support up-to-date in any distro

I am not aware of distro maintainers complaining about this, or at least, not any time in the last few years. Do you have some context to share? We want distros to have a good experience.

Debian unstable (IIRC) updated Firefox a couple weeks late because cargo got a couple of new dependencies and nobody added them to the Debian repos. This is because they forbid static linking, so there are hundreds of packages of popular Rust crates compiled as shared libraries. Even worse, some of them have multiple variants, for various combinations of features.

Sure. This is one instance of this happening, and it was fixed, it just took some work. Nobody even filed an upstream bug. This isn’t something exclusive to Rust. It’s almost arguably more due to the way that browsers are treated as an exemption from distro packaging policies (which I appreciate, personally.)

That’s a Debian issue. They’re trying to foist a dependency management strategy intended for C on a different language with a different strategy

> Comparing coreutils to a self-hosting compiler is a strained analogy.

What? Why? Unix is a self hosting environment. You can't even boot the OS you're building on without coreutils or the equivalent.

> Fresh hell, they all seem to be build dependencies of each other. ‘sed’ is its own build dependency.

Well, they are the "core" utilities after all. This becomes less of an issue the further out in the ecosystem you get.

> at the exact versions and in the exact locations on the filesystem that the build tooling is willing to look.

This isn't strictly true.. you just need _compatible_ versions to be available, and most build tools and even the compiler itself will check several commonly used locations for code.

There are a few corner cases, but by and large, this isn't as big a problem as it might be for other languages that make writing incompatible interfaces orders of magnitude easier than with C.

> Specifically, hasten the day where my entire application dependency tree doesn’t bottom out in some Autotools, CMake, shell script, project.

Each those solve the same problem at different levels. Perhaps this is because there is no _one true_ way to replace them, and the advantage is that each project uses the tool most suited to their case.

Perhaps the built-in tools with these "modern" languages are a good general tool for the majority of developers, perhaps they aren't. If they aren't, then now you have the same problem all over again in a new ecosystem.

I don’t know man, I’ve written a lot of C and C++ and other languages and the sheer effort to get your average C project to compile is much greater than others. The dependencies aren’t listed anywhere, they are rarely compatible with the versions available via your package manager (so now you’re building more things from source), ./configure regularly runs into problems (“couldn’t guess which platform you were on”, “your version of grep is incompatible”, etc). It’s just a big mess of yak shaving. If you think I’m making too big a deal, I invite you to spend some time in the Go or Rust ecosystems where you just “cargo build” or “go build” and you’re done. You don’t need to understand anything about the project, there are no shitty DSLs to grok (looking at you, CMake). Things just work.

With CMake you can build statically in a very reproducible cross platform way if you use Hunter and toolchain files.

the tools are all there - just they're not popular and aren't widely used

I wrote a bit of an old intro to the workflow: https://geokon-gh.github.io/hunterintro.html

This is an example of a project that's well setup using this pattern: https://github.com/elucideye/drishti

And having things work dynamically ... well that's what package maintainers due. It seems inherently very fragile. Developers only build test and develop against whatever dependency versions they have locally. you can't test against every version under the sun

Regarding build tools for C, you might find this one interesting:

https://github.com/vmchale/cpkg

Yeah, there are lots of good build systems for C (didn’t know about this one; will have a look; thanks for sharing), but the problem is that they have tiny adoption, especially among “core” ecosystem packages, like coreutils and friends. The problem isn’t technical, it’s political and cultural. This ecosystem is hostile toward its users for no discernible reason.

If anyone knows of a distribution that aims to use only software with sane build systems, let me know, I would be very curious.

Bootstrapping and too many binary dependencies at the early stage is a known problem in GNU Guix and there are people working to fix it, such as the GNU Mes project to bootstrap the system using a minimal compiler, shell and set of utilities written in Scheme. Despite this, there are still many steps to bootstrapping a full system and the complete process is not simple.

It's not for "no reason", in general you will not be able to get away from having a complicated build system at the lowest level. Otherwise you would need to be prepared to give up on nice things like cross-compiling and supporting more than one compiler, CPU architecture, etc.

Actually focusing on cross generally usually forces when to be bootstrapping better.

The problem is most distros / deployed systems are managed and bootstrapped imperatively, which there is nothing pushing back against complicating the bootstrapping. Thus, entropy wins and it gets more complex.

As someone who as actually started about reproducing BSD bootstraps declaritively (https://github.com/NixOS/nixpkgs/pull/82131/files) they are way more entangled than they need to be, so Theo de Raadt has no ground to stand on.

Nix is becoming more of a name in dependency management systems: https://blog.galowicz.de/2019/04/17/tutorial_nix_cpp_setup/

Its claims to fame are support for multiple languages, reproducible builds, and system-wide package caching.

Yeah, Nix is really neat. It's the right direction for sure, but it's mostly used as a wrapper around autotools and cmake projects by third parties and not so much as a first-class build tool by the maintainers of C/C++ libs (this isn't going to change because Nix prefers to be "innovative" and unfamiliar in just about every design decision--its core approach to reproducible builds is right on, but the execution leaves a lot to be desired if you want to convince a community of developers to adopt it as a build tool). So anyone writing a Nix expression to package an autotools or cmake project (me, in this case) still has to deal with their nonsense.

Nix is a package manager, not a build tool. It's never been in-scope for it to figure out how to invoke a particular compiler or linker. All it can do is make it easier for you to invoke another build tool, of which there are quite a few.

I would not recommend the use of autotools in new projects, but in my experience CMake with Ninja is fine for distro maintainers to deal with — All other modern build tools I've seen are roughly the equivalent of it. If you find that it's hard to get over the hump with that, then I hate to say it but I don't think solutions will come easily elsewhere on any OS or platform. The state of things on free software operating systems is constantly improving but for now it's something you have to get used to. There are no silver bullets here.

> Nix is a package manager, not a build tool. It's never been in-scope for it to figure out how to invoke a particular compiler or linker. All it can do is make it easier for you to invoke another build tool, of which there are quite a few.

I agree that Nix is focused on package management and not building, and that’s a bummer because Nix is perfectly capable technically. In particular, Nix can trivially invoke a compiler or linker; its issues are that it assumes that it will only be used by a relatively small number of package maintainers instead of a much larger number of developers, so it prefers to be novel and innovative instead of familiar and straightforward. This makes it really hard to get developers to buy in, but there’s no technical reason why it can’t be used as a build system.

> I would not recommend the use of autotools in new projects, but in my experience CMake with Ninja is fine for distro maintainers to deal with — All other modern build tools I've seen are roughly the equivalent of it. If you find that it's hard to get over the hump with that, then I hate to say it but I don't think solutions will come easily elsewhere on any OS or platform. The state of things on free software operating systems is constantly improving but for now it's something you have to get used to. There are no silver bullets here.

The silver bullets are getting away from the C ecosystem to the greatest extent possible, otherwise you will have to build way more autotools and CMake projects and this isn’t a good use of anyone’s time. As fun as C is, for most applications, we have better alternatives nowadays. They aren’t perfect, but they let you move a lot faster especially with respect to building software.

> It's never been in-scope for it to figure out how to invoke a particular compiler or linker.

True

> Nix is a package manager, not a build tool.

False. Nix should be the go-to tool for executing and caching all dependency graphs. Just the planning of those graphs is domain-specific and out of Nix's purvue.

Build systems that use Ninja actually get the execution-planning separation right. (Try Meson if you hate CMake.) So basically, we just need to make Nix better than Ninja in every way, and then hook it up to these build systems (directly or with ninja2nix).

Of course, then we will demonstrate all the impurities and non-determinism in people's CMakeLists. That alone is the cultural problem to solve for normie evangelizing. Everything else is technical and there's use bemoaning Nix is just too weird.

Python's build ecosystem IS sane and well-designed. It's the pinnacle of package handling so far. A separate package manager which works together with a separate build system which still works when a distribution wants to package a python package.

What build systems like cargo seem to think is awesome is probably the most backwards idea of build systems I've ever seen. Static linking and automatic downloading of dependencies. Even the ability to directly pull from a github repo. No wonder almost nothing written in rust ever gets packaged.

Dependencies are hard, following a spec like semver seems to have confused people. Developers are lazy and don't want to think about settling down their API. Solution? Just give up and encourage bad behaviour. Thanks guys. Great solution. I think I'll stick with the previous one.

> No wonder almost nothing written in rust ever gets packaged.

rustc is also built with cargo, and it is packaged on most distros. A lot of distros are including packages of other tools too.

> Python's build ecosystem IS sane and well-designed. It's the pinnacle of package handling so far.

Big oof. Python packaging has a lot of problems, and defaulting to a single shared location when installing, that might conflict with your default package manager, is very high on the "What the fuck are you even doing" list of issues. You also very easily get into version hell. Unless you start using virtualenv, at which point you're basically doing the same thing cargo is.

> automatic downloading of dependencies

How is that a problem? Cargo has a lockfile to ensure the dependency downloaded is the exact same, various flags to prevent cargo from updating the lockfile (--frozen and --locked), and even the ability to vendor dependencies with cargo vendor[0]. It defaults to the most convenient things for developers because packages will be worked on many more times than they are packaged.

> Even the ability to directly pull from a github repo.

So does python/pip. And most package managers I've worked with. Crates.io will refuse packages that have github dependencies though, so this only comes up when packaging something that isn't really ready to be packaged.

As for linking, there are several rust-specific problems to dynamic linking:

1. Rust currently has no stable ABI (apart from the C ABI). This means updating rustc (and maybe even updating llvm) may cause the ABI to change, and libraries to become incompatible. There is currently no plan to fix this on Rust's part, and in fact some opposition, since it would prevent some optimizations from happening (such as niche-filling optimizations).

2. Generic functions and impls would not be part of this, as monomorphised variants would have to end up in the final binary. I believe C++ has the same problem with templates. A lot of Rust APIs are generic. This could theoretically be fixed (at least partially) through automatic promotion to dynamic dispatch, or other similar schemes.

[0]: https://doc.rust-lang.org/cargo/commands/cargo-vendor.html

> Python packaging has a lot of problems

Sure, but I think that what systems like cargo provide is worse across the board.

> defaulting to a single shared location when installing, that might conflict with your default package manager, is very high on the "What the fuck are you even doing" list of issues.

This is easily solved if pip was to have a way of listing all dependencies which it's about to install. Then I could mangle the names to fit my package manager's style and automatically install them with my package manager. But I will give it to you, this omission is one of the few things which pip could improve on.

> You also very easily get into version hell.

This is not a pip problem. Stop using things which result in dependency hell. Semver solved this problem a long time ago. If developers insist on not stabilising their APIs and breaking them all the time those developers libraries' shouldn't be used.

> Cargo has a lockfile to ensure the dependency downloaded is the exact same,

Great, now I can make my software depend on an out of date and vulnerable version of a library. Why do you think encouraging bad behaviour is a good thing? Yes, getting versions right is hard, but it's better than the alternative of static linking and having to update 100 pieces of software when you find a critical vulnerability in <popular network library>. This gets even worse when not only does every developer have to change this lockfile to update their code but now because they used lockfiles and as a result the upstream library developers felt it was safe to break API 100 times between when that library was locked and when it was patched. Now a simple update of <popular network library> which would involve pushing an ABI compatible patched version via distribution channels requires 100 developers to all push updates to their software which may or may not include having to re-write portions of their software.

> It defaults to the most convenient things for developers because packages will be worked on many more times than they are packaged.

Only because you allow and encourage it to happen. Which I hope I've made my point clear is NOT a good thing.

> So does python/pip. And most package managers I've worked with. Crates.io will refuse packages that have github dependencies though, so this only comes up when packaging something that isn't really ready to be packaged.

So it's good to see that this issue has been appropriately addressed. I wasn't aware that this was a restriction of crates.io. That being said, it doesn't stop everyone and their grandmother from just not using crates.io for certain things and not having their stuff on crates.io.

I wasn't aware this was possible with python but I also never see python packages using it.

> Rust currently has no stable ABI (apart from the C ABI).

Which isn't a good thing.

> 2. Generic functions and impls would not be part of this, as monomorphised variants would have to end up in the final binary. I believe C++ has the same problem with templates. A lot of Rust APIs are generic. This could theoretically be fixed (at least partially) through automatic promotion to dynamic dispatch, or other similar schemes.

I think Ada has a solution for this. Not 100% sure but it can't be impossible to solve if only rust developers made it their goal to make libraries work somehow.

Rust not having a stable ABI is not universally bad; it allows for breaking changes obviously. Swift has stabilized its ABI with generics, but its implementation is not quite zero-cost so I am not sure if Rust could go that direction.

Python packages don't interact well with the built in package manager on my os. It doesn't even try to, which is a major design flaw.

Now if python had a way to do this and package managers refused to interopate (as I believe would happen) we would have a different discussion.

Pip doesn't need to integrate, it just needs to install elsewhere or give you a way of listing what will be installed. Cargo on the other hand just doesn't even bother letting you try to use it with a distro package manager.

Reply on news.ycombinator.com