Hacker News Reader

Comment by tedunangst | Hacker News Reader

I have never shipped unix software that used the system package manager, for a number of reasons. If you want it to be cross platform, you need to build/test/ship the fallback tgz manual install anyway. Packages don't save any work, they only increase the size of your build farm and testing matrix. The software was usually built against specific versions (often modified) of 3rd party libraries. We can't rely on the system to have them, so we need to ship them too, so it's not like relying on system packages would reduce our download size at all. All told, the number of customers who care is very small, either because people just don't care or because the people who care don't pay for software.

That was for commercial software of course, but even for open source software, I'm not sure why it's the author's responsibility to go setup a dozen different VMs. That really is the package maintainer's job.

View on HN

* The software was usually built against specific versions (often modified) of 3rd party libraries. *

That sounds like a disaster waiting to happen... maintaining a different version of libraries outside of their main development trunk means that:

1. Extra steps when a new main version comes out - you have to repatch your differences in.

2. #1 incurs a delay, thus a longer time that your customer are running without patches, which may be a security issue.

3. You won't do #1 at all. Thus you become reliant on old, unmaintained, buggy, security-holey libraries.

The only time I can think this is acceptable is to fix problems that upstream won't fix, or are platform specific (ie, not with your custom software, but with the OS underneath everything).

I'm not the OP, but I will respond anyway:

1. We have made changes to the 3rd party source code that we want to ship now, not when they finally let the patch into their source tree.

2. We get new updates from the latest release, patch in our stuff and we are set, quick test on the CI servers and we can ship. Yes, customers may be without the fix for a little while longer...

3. It is actually in our best interest to keep 3rd party libraries at their latest release, if we want secure software. Which is why it is part of our release documents and steps that are required to be followed before we release.

Then what you have is a fork of the library. There is no reason you can't distribute forks of the library (to either your own paths or with your own modified names) using the system package manager.

If you are expecting the person taking your patches to install them to the system, however, that's a whole 'nother ball game, and gets back to the original point: as a system administrator, I have no reason to believe your "better" versions of those libraries are going to interoperate with everything else on the system, and I am then going to have to do a bunch of work doing that isolation myself.

And if you allow the customer to upgrade libraries themselves (or install any version they desire) you'll risk breaking the application. That can be even worse.

Patching in new versions isn't usually a huge deal, it's just not worth the trouble. Let's say we're using version 1.2, and we fixed a few bugs in it over the past year. Now version 1.3 is out. Should we upgrade?

Maybe 1.3 fixes a few bugs we haven't encountered yet. But that's unlikely. We've been using it for a year. Far more likely, as in practically a certainty, is that 1.3 will introduce some regression that won't be discovered until it affects some customer.

Not every bug in every program is a security vulnerability, and framing the issue that way distorts things.

Well, if you had uptreamed your fixes against 1.2, they would be in 1.3, wouldn't they ? I find it a poor excuse for not upgrade assuming API and ABI are stable.

It's just laziness. It's easier because that way you don't have to argue with Management. But you also don't educate on the necessity of fixing security issues. Hooray for 1.4 JVM still in production for instance. And it's perfectly ok. Until some breach happens .....

Not every upstream will take your patches. Not every upstream can take your patches.

Getting my fixes into 1.3 doesn't prevent 1.3 from having regressions.

> I'm not sure why it's the author's responsibility to go setup a dozen different VMs

To make sure it compiles and works under dozen different distributions (in practice centos + ubuntu/debian usually cover most cases). If you don't test it, how do you know it's usable at all? Besides, as long as you don't invent some crazy custom build system, the only thing you really have to write is metadata and dependencies (rules file can be 3 lines long for autotools software).

Also, this way you can just push your software to a build farm like launchpad ppa and let them do the dirty work. If you include self-contained tests with the sources, you can even run them after the build to verify correctness for that environment. This is much better than doing it by hand.

Also keep in mind that the first person to deploy your software automatically to a couple of places will either use your/distro's package or will have to make their own. Building from source works only for one-person uncontrolled environment.

Please add FreeBSD to the list of OS's you use for testing of compilation.

You have no idea how frustrating it is to have an open source project claim to run on unix like operating systems and not compile correctly on FreeBSD or Mac OS X...

try using openbsd sometime... :)

As much as I agree... I'm not sure how much it's worth the effort until someone points out that your project definitely doesn't work. Even looking at silly stats like: http://distrowatch.com/stats.php?section=popularity - covering one popular rpm and one deb distribution, you're covering majority already (with only slight modifications if necessary). Mac users will expect their own packages anyways and you can't run Mac OSX legally without owning the hardware, which many developers don't own.

Regarding FreeBSD... lack of any good result in the first 3 pages of "freebsd packaging guide" search should explain a lot.

search for "freebsd porting guide".

Ok - I wasn't clear on that one. I know they're called ports and would probably look for stuff like port creation / porting / port-something-else. Yet, it seems a bit weird to me that looking for freebsd packaging guide doesn't give you anything even remotely useful. (consider some developer who has a couple hours to spend, thinks about making a freebsd port, looks for packaging information and concludes that it's not even documented / not worth learning right now)

In FreeBSD to create a package you create a portfile, the packages come free at that point ...

But I understand your concern.

Generally if it compiles on FreeBSD it compiles on Mac OS X, at least that has been my experience in the past. I am in the unique position that I develop on Mac OS X, then do my tests on FreeBSD and when it works on those attempt it on Linux.

The counter-argument is that testing on more platforms tends to expose bugs that it's worth fixing because they would have affected you anyway, you just didn't know it yet.

Imagine I'm your average, workaday software developer.

If I've written some software to scratch a personal itch and I push it out to github because I reckon someone, somewhere might benefit at some arbitrary future point, or to show off to a future employer, or whatever, then I really do not care about making sure it compiles and works on a dozen distributions. Just getting it working to fix my problem, then publishing it, ticks all the boxes I'm interested in. After all, I've got fifteen other problems I need to solve this week, none of which involve spooling up VMs for distributions I never use.

If a packager then decides that their favourite distro would benefit from having my code packaged nicely, at what point does that become my responsibility? It Works For Me(tm), and beyond reviewing and applying patches other people might send me to improve it, I don't necessarily have any interest whatsoever in what happens outside my repository as long as it doesn't stop the code from solving my problem. It's possible that I might be both developer and packager for a distribution or two, but that's the exception, not the rule.

Personally, I think the right way is for a clear path from language-package to distro-package to be provided by my environment such that I don't have to think about it, and I think that the bulk of that responsibility should be shouldered by the distro - but it's insane to think that the language environment shouldn't help, because there is a mutual benefit there. I think we're nearly there with RVM and rubygems, but that's about as far from autotools as you can get.

> I'm not sure why it's the author's responsibility to go setup a dozen different VMs. That really is the package maintainer's job.

If you're in the enviable position of having lots of people willing to put in the time to maintain your package, and a distro that thinks your software is sufficiently noteworthy to include in their distro. But if your software is relatively new and you want to encourage wide adoption, one way to help is to make the install process friendly and familiar for sysadmins as well as fellow developers. And if you speak their language that goes a long way towards building that trust.

I'm not sure why it's the author's responsibility to go setup a dozen different VMs. That really is the package maintainer's job.

Agreed. I do, however, expect upstream to accept reasonable patches from package maintainers to fix building bugs or make it easier for people to build packages themselves.

For the few times I've done it, I've had no trouble getting upstream to accept an updated .spec file (or even a barebones .spec file) that they can distribute in their targz to make it easier for the people who want to build their software as a package if their distribution doesn't provide one.

In lots of cases the separate package maintainer's do a good but ultimately thankless task. The example I gave between the rabbitmq packages which are built as part of the development effort and made available on the rabbitmq site and the solr packages which are sometimes out of date and not referenced by the site highlights the point I'm trying to make.

Reply on news.ycombinator.com