Like if building a pet app is your company/team's job then containers more than fine. If you're a distro maintainer that has thousands on thousands of packages then sharing libs starts to make a lot more sense.
They're screwed. Hopefully you aren't.
Hopefully.
The problem is that distributions get their software from upstream developers. And if upstream decides to go static or use embedded code copies, the maintainers have much more work trying to untangle everything.
In my experience, packaging Go software for Linux distributions is very frustrating and tedious because of that.
Packaging Python or C/C++ software is much easier.
Wondering since the application developers I know seem to prefer static linking.
I can't help but wonder if we'd all just benefit from better build systems, so that "rebuilding all downstream consumers" wouldn't be hard.
Perhaps this "untangling" isn't the best idea, at least not always?
- the bundled things might have unclear licensing
- bundling stuff that is available in the repos is sub optimal, as the repo version will be updated for security issues while the bundled version will not
- you can use some mechanism to mark what stuff is bundled in a package, but then you need to make sure the bundled thing is patched and rebuild each thing bundling it, where with system wide dependency you just update that and you are done
I'm fairly sure that almost all projects are receptive to updates for dependencies with security issues; I don't see why a distro needs to "untangle" anything for them.
> with system wide dependency you just update that and you are done
Unless the system-wide update won't work with your program.
The entire point of the Go tooling is that you can reliably and consistently build your software, and replacing random parts to create a weird unsupported (by the actual developers anyway) version is something the Go tooling was never designed to support: it goes exactly against what it was designed to do. Hence my previous comment: perhaps this isn't the best of ideas.
- Outdated packages
- Having to build from source when the package is not in the repos
- Failed builds for no good reason
- Unability to have many versions of that same packages
- having to install log(n) build tools if I want to build n packages
- etc
while in half-developed OS'es everything just works (application-wise, at least)
There's a fundamental unresolvable tension in automatic update systems between "not updating breaks things" and "updating breaks things"; you cannot solve for both at once and either one will get users mad at you, although not upgrading at least gives a pushback argument. See how all this has played out with Windows and Mac updates, for example.
You really should check what APIs your dependencies provide and how stable that API is, then set minimal dependency version. If you set the dependency to a specific version, you are creating a headache for anyone trying to use your software later on (by potentially forcing them to use oudated and insecure software) and for distro maintainers trying to keep distro software up to date and secure.
What if the dependency is so unstable that you have to pin the version or even use custom patched version? Well, thenaybe its not somethong you should be depending on & rather use something with less features but more stable. Othervise you are really being unresponsible - using quick convenience at the cost of long term usability of your software.
The thing is that in general you don't want to have many version of libraries available in the distro in parallel as each has to be separately maintained, security patches applied, build issues fixed, etc., eating valuable maintainer time. Also in some cases libraries of different version can't coexist on a system cleanly.
Then imagine every piece of software just pins versions of their dependencies to a specific random version that happened to work at the time. To satisfy those arbitrary dependencies the distro would have to maintain all these versions at the same time, which is simply impossible resource vise, not to mention incredibly wasteful (both in maintainer time & system resources).
As for stuff breaking if you don't pin dependency version - well, distros have mechanisms to handle that. For example for Fedora, there is a stable release very 6 months & stable releases are not expected to get major changes in libraries, just bug fixes and smaller enhancements.
And at the same time there is a rolling version of Fedora called Rawhide, where all the latest package versions land and where integration issues are addressed. So any breakage would happen on Rawhide and be addressed by maintainers (of the library/software affected or both) long before a new stable release is cut from Rawhide and users will actually use it.
For an example I'm maintaining the PyOtherSide Qt 5/Python bindings on Fedora. A while ago the build failed due to Python being updated to 3.9. I reported the issue upstream, which quickly fixed it and I've built an updated version in Rawhide. All this long before a stable Fedora version will get Python 3.9, but I can be sure that when this happens, all will work fine.
[0] Remember when Debian generated predictable random numbers because a maintainer wanted valgrind to shut up?
Now in comparison people using NPM just blindly pulled random stuff directly from upstream without anyone doing any sanity checking at all - no wonder one package vanishing made the whole thing fall over, often directly in production.
Especially with Python, and especially with the 2/3 split, people got used to assuming that the distro version was something broken to work around (e.g. Redhat, OSX), and that all "real" work happened in one's local language-specific package cache or venv.
I'm increasingly of the opinion that it's a mistake for distros to ship Python or Ruby packages in their distro-specific package format, but I can also see that's going to be a holy war.
As Linux user who sometimes build packages (tinker around) I need all kind of dependencies - python, ruby, perl, haskell. It is much safer and faster to use distro packages.
Breaking changes on major version is awful for any consumer. Python 2/3 story is a shame. These can't be arguments against distro packages.
I can't help but wonder if there's a better way that could scale to open source. I personally like monorepo-based development a lot - where you have one version of every library for the whole repo, and a total ordering on changesets (and a strong test suite to catch regressions). But organizing open source into a monorepo or even a virtual-monorepo seems tricky. Might be the sort of practice that can work well for companies but not so much when decision power is more distributed.
And users would find outdated versions (maybe unmaintained). Current system update dependencies automatically or allow collaboration on life support (patched application, dependencies, config). It would be good to have alternative like container or virtual machine with snapshot from the days long past. But it should be clear this is not safe.
Em, like all other packages?
Here are the scenarios:
- dynamic responsive remove bug: Positive/neutral. Team X would have done it anyway.
- dynamic unresponsive remove bug: Positive.
- dynamic responsive add bug: Negative. Team X will see the bug but only be able to passively warn users not to use Y version whatever.
- dynamic unresponsive add bug: Negative. Users will be impacted and have to get Y to fix the error.
- static responsive remove bug: Positive/neutral: Team X will incorporate the change from Y, although possibly somewhat slower (but safer).
- static unresponsive remove bug: Negative. Users will have to fork X or goad them into incorportating the fix.
- static responsive add bug: Positive. Users will not get the bad version of Y.
- static unresponsive add bug: Positive. Users will not get the bad version of Y.
Overall, dynamic is positive 1, neutral 1, negative 2, and static is positive 2, neutral 1, negative 1. Unless you can rule out Y adding bugs, static makes more sense. Dynamic is best if "unresponsive remove bug" is likely, but if X is unresponsive, maybe you should just leave X anyway.
If the install is on a consumer machine for regular usage then the right answer is shared libraries for the machine. It adds a lot of complexity for the packagers for that OS but you get a lot of safety for that consumer.
If the install is an inhouse app deploying to servers a company controls/rents whatever then the right answer is probably a static application. Consumers are more likely to want to chase the latest version of everything. Company developed software is far less likely to want that. The problem then becomes less about packaging and more about deploying fixes quickly. A static binary that is rebuilt, tested, and deployed is going to be a smoother path to fixing that error than figuring out how to deploy your new shared library and avoiding any issues in our production environment caused by clashes to that library. This is made more problematic by the likelihood that you will need multiple versions of that shared library with the fix to meet the needs of applications that can not yet be upgraded. Static binaries make things easier in this scenario and thus quicker to resolve.
Sure people may not use the tools they have but then again people also don't upgrade their OS packages or maven packages. So for those people packaging everything doesn't hurt any more since they've got rampant CVEs already.