Hacker News Reader

Debian has a terrible track record. Just look at the OpenSSL/Valgrind disaster. As a former upstream developer myself (on the Wine project), all Linux distros found unique ways to mangle and break our software but Debian and derived distros were by far the worst. We simply refused to do tech support for users who had installed Wine from their distribution the level of brokenness was so high.

You may feel that developers are some kind of loose cannons who don't care about quality and Debian is some kind of gold standard. From the other side of the fence, we do care about the quality of our software and Debian is a disaster zone in which people without sufficient competence routinely patch packages and break them. I specifically ask people not to package my software these days to avoid being sucked back into that world.

As a sysadmin you shouldn't even be running Maven. It's a build tool. The moment you're running it you're being a developer, not a sysadmin. If there are bugs or deficiencies in the software you're trying to run go talk to upstream and get them fixed, don't blame the build tool for not being Debian enough.

I find it weird that you consider 'packaging' to be something a sysadmin should do, but 'building' to be something they should not do. Aren't they both forms of 'prepping code for use'?

And then state that you don't want your own software packaged. So, if a sysadmin is not allowed to build and not allowed to package, how are they supposed to get your code into production? "curl foo | sh"?

I read that the other way around. He specifically asks people to refrain from packaging his software. I think the implicit implication is that he does want sysadmins to run his carefully constructed build scripts in order to install the application.

I don't consider packaging to be a sysadmin task. On any sane OS (i.e. anything not Linux/BSD), packaging is done by the upstream developers. That doesn't happen on Linux because of the culture of unstable APIs and general inconsistencies between distributions, but for my current app, I am providing DEBs and woe betide the distro developer who thinks it's a good idea to repackage things themselves ...

Well, we're going to have to agree to disagree there, because I think Windows packaging is fucking insane.

One of the things I loved about my move to linux and .deb land was that if I uninstalled something, I knew it was uninstalled. I didn't have to rely on the packager remembering to remove all their bits, or even remembering to include an uninstall option at all. Or rely on them not to do drive-by installs (which big names like Adobe still do, out in the open). And not have every significant program install it's own "phone home" mechanism to check for updates. The crapstorm that is Windows packaging is a fantastic example of a place where developers love and care for their own product, but care not a jot for how the system as a whole should go together.

I don't know agree with that.

Debian feels like a distribution maintained by a bunch of sysadmins: People who have shit to do, and who understand that the purpose of a machine is to get stuff done, not to run some software.

A lot of sysadmins believe since they are responsible for the software, they need to be able to build stuff and fix some stuff themselves (i.e. it can't wait for upstream). In my experience, it's usually something stupid (like commenting out some logspam), but it's critical enough that I can imagine a lot of shops making it mandatory to ensure they can do this.

Really proactive sysadmins do try to run fuzzers and valgrind and do try to look for bugs rather than waiting for them to strike. And sometimes they get it completely wrong, as in the OpenSSL/Valgrind disaster, but they usually ask first[1].

Now I don't agree with everything Debian do, and I don't want to defend everything they do, either, but I think programmers in general need to get out a certain amount of humility when dealing with sysadmins: Because when these sysadmins say that they're not going to package hadoop because the hadoop build process is bullshit, it isn't appropriate to reply "well you guys fucked up openssl, so what do you know?"

One thing that would help is if we didn't look at it as Programmers on one side of the fence and Sysadmins on another side. Programmers have problems to solve, and sysadmins have problems to solve, and maybe you can help each other help solve each other's problems.

[1]: http://marc.info/?l=openssl-dev&m=114651085826293&w=2

You really nailed it. Among my duties are systems administration for a company that works with a lot of software development vendors. We have a user acceptance team that makes sure that we get what we ordered, that the QC stays at a high level. So functional problems, that's their deal. But they're not sysadmins, they can't easily see what developer choices make administering the servers more complicated, more fragile, more expensive, or more insecure. This shifts my job from the end of the process (here, run this!) to the beginning (hey guys, let's use these tools instead, it'll make everyone's lives easier).

As such I'm very pro containers as they will eliminate a ton of deployment effort and allow me to manage different environments much more easily. But it means that there needs to be a much bigger magnifying glass on the container contents early in the process as opposed to the moment of deployment.

> The value of debian is not that they package (or repackage) everything into deb files but that they resolve versioning and dependancy conflicts, slip security fixes into old versions of libraries (when newer version break API/ABI), and make it possible to integrate completely disparate software into a system.

Maven has exactly the same capabilities as deb does - you can depend on versions, depend on a range of possible versions, exclude things that conflict and so forth. And it puts even more emphasis on fully reproducible builds (with the aid of the JVM) - in that respect it's closer to nix than apt.

> But the sysadmin does feel responsible. He is responsible for the decisions you make, so if you seem to be making decisions that help him (like making it easy for you to get your software into debian) then he finds it easier to trust you. If you make him play whackamole with dependencies, and require a server (or a container) all to yourself, and don't document how to deal with your logfiles (or even where they show up)

Wow, self-important much? Too many sysadmins seem to forget that the system exists to run the programs, not the other way around.

> If you make him play whackamole with dependencies, and require a server (or a container) all to yourself, and don't document how to deal with your logfiles (or even where they show up), how or when you will communicate with remote hosts, how much bandwidth you'll use, and so on

On the contrary, maven makes the requirements much simpler. I have literally one dependency, a JVM, so it can run on any host you like (no need to worry about shared library conflicts with some other application). It needs to download one file (shaded jar) from our internal repo, and execute one command to run it. That's it.

> That's what Maven is. It's a surprise box that encourages shotgun debugging and using ausearch features to do upgrades.

No, it's just the opposite. All the dependencies and project structure are right there in declarative XML. It's what make should have been.

> No, it's just the opposite. All the dependencies and project structure are right there in declarative XML. It's what make should have been.

When make was written most machines would have just exploded at the sight of a typical build.xml, and downloading tens or hundreds of packages from anywhere was simply out of the question.

Also, 'dependency' means something completeley different in make as opposed to maven - I don't think modern build systems do even care much for make-style deps.

> When make was written most machines would have just exploded at the sight of a typical build.xml, and downloading tens or hundreds of packages from anywhere was simply out of the question.

Sure. But the notion of doing things declaratively existed (Prolog predates make by five years). And the biggest difference between make and the scripts that preceded it is that it's more structured, with a graph of targets rather than just a list of commands.

If you add the ability to reuse libraries of targets (something that sort-of exists via implicit make rules), restrict targets to something a little more structured than random shell commands, and - yes - add the ability to fetch dependencies (including target definitions) from a repository, you end up with something very like maven.

> The value of debian is not that they package (or repackage) everything into deb files but that they resolve versioning and dependancy conflicts, slip security fixes into old versions of libraries (when newer version break API/ABI), and make it possible to integrate completely disparate software into a system. They also have a great track record at it.

The analog of the aspects of what Debian does that you're talking about here in the HStack world are companies like Cloudera, who, surprise, make their stuff available as debs and PPAs.

Building your own Hadoop from source and complaining that the resulting product is unvetted is sort of like doing a git pull of all the Linux dependencies and building that.

But the problem is that the ones doing the vetting (i.e. Debian) have given up on making a vettable distribution because the build is so broken.

But Debian isn't the Debian of Hadoop. Cloudera is.

Why should we assume the Debian Foundation is the sole trusted source of every type of software?

What you are saying makes no sense.

And yes, if I'm using Debian and didn't add any PPA or extra sources, then the Debian Foundation IS the sole trusted source of software. And you do that because you know hey won't fuck up the system, which (and that's the whole point of this thread) the others certainly don't.

Now Debian is telling you: we see now way to distribute this software and guarantee what you are getting or that it won't fuck up the system.

Do you think I'd consider installing that junk?

So everyone who runs Hadoop is installing junk? Seems like plenty of other companies have been able to build businesses on it without adhering to your Debian-only rules...