Debian Packaging from First Principles | Hacker News Reader

Debian Packaging from First Principles

mikecoats.com

87 pointsalexzeitler26 comments

Isn't this overly complicated for what amounts to what could be a single file with 7 lines of information, instead of 3-5 tar archives that contain a single file with 1-3 lines of information?

but still, that was a pretty interesting write up.

Guessing that this is the result of years (or decades?) of incremental changes maintaining backwards compatibility.

I wouldn’t agree that it’s overly complicated. Compare it to the Docker/OCI container format with multiple layers of tarballs and complicated manifest.json files. The deb format appears much simpler.

Docker is just like Debian in this instance. It would give you a base system from where you create the package using the same exact steps.

Yeah, Debian is very, very unergonomic for package maintainers. I get that it's historically groen, but it has historically grown into an abomination that scares off people.

Arch packaging is a breeze in comparison. You indeed only need one file and that's it.

Well, to be fair, no package maintainers are touching any of the stuff described here. All the "ar" and "control.tar" stuff is handled for you by the tools. This isn't a "this is how you build packages" article, it's a "here's how the guts of these packages work" article.

to be really fair, the tools package maintainers use are more complicated than stuff described in the blog post.

Best way to discourage anyone from generating a .deb package is recommending him read the official "new maintainers guide"

To do the stuff the blog post talks about you 'dpkg-buildpackage'. That's it.

After years of maintaining packages, I think I decided that much of the complexity is unfortunately warranted. You want stuff to work in all sorts of situations, with all sorts of different other things installed at the same time. You want upgrades to work. You want tests to work and to run, but probably you want to skip them when cross-building. You want the package to be co-installable across architectures. You want the package to be cross-buildable and to be usable to cross-build other packages. And you want your builds to be reproducible. And lots and lots of other things, all the while having to deal with upstreams that often don't care about many of these things.

I haven't seen any other packaging system that is both nicer and maintains Debian's high standards. Maybe something exists. I think Debian's biggest problem in this area is the documentation. The official docs should be more accessible, but that's a big job that nobody has stepped up to do. Yet.

I agree completely. On the other hand, it should be recognized there's a difference between packaging for the official repository and packaging done by individual users/sysadmins/developers for distributing independently. Unfortunately all of Debian's tooling and documentation is for the first scenario.

I must emphasize, I'm not saying, to quote you, that the documentation should more accessible, but nobody has stepped up to do yet. I'm saying, they're completely fixated in their approach to tailor packaging tools and documentation for creating packages for inclusion in official repository. Debian's tools and documentation for creating ".deb" packages are completely intermingled with their policies for including packages in the official repository. That's where all the complexity comes from. They're mixing the policy (Debian repository rules) and mechanism (deb format), and unless they recognize this, no amount of packaging tools or documentation improvements will solve it.

Which is such a shame by the way, because like you said, Debian packaging ecosystem is really powerful and would render a lot of complex machinery devops people use today unnecessary. You can distribute out-of-band updates to data files, push configuration (instead of configuration management) [1], do deployments with virtualenv/gems included (instead of containers) [2], proxy/partial repo mirrors for staged updates/deployments etc. Unfortunately if you decide to it, you'll either have to go with the Debian sanctioned way and tear your hairs because of unnecessary complexity, or you'll go look for 3rd party tools and hunt for blogs posts all over the internet as substitute documentation.

[1] https://wiki.debian.org/ConfigPackages [2] I recommend checking out Vincent Bernat's excellent "Pragmatic Debian Packaging" series (https://github.com/vincentbernat/pragmatic-debian-packages) for some examples

I don't see Debian as any more arcane than RPM, SVR4, BSD packaging.

Barely anybody uses Arch outside the HN bubble, it's a blip compared to the amount of Redhat/Debian out there in Enterprise/Web 1.9

Yes, 1980s/early 90s tooling is mostly similar in the amount of effort it put into optimizations that are simply irrelevant today. And it still works, if you're a fat lazy enterprise that can afford much, much more horrible inefficiencies than those rpm/deb tooling inflicts on its users.

But that doesn't make it superior, or something to be proud of.

Not sure if you're arguing against common packaging systems still in wide use, or arguing for more modern packaging systems (arch, nix, docker, flatpak, snaps etc). The latter will all "win" on their merits at some point and become fat and lazy themselves as they evolve and grow, or fail to gain marketshare and wither. For now the fat lazy ones hold the lion's share, they are what Enterprise customers are used to dealing with, because those companies prefer reliable, predictable and boring. Obviously you can use whatever you want in your startup, 95% might be here now but probably gone tomorrow.

Really, I'm not sure what the point of your posts is. You don't like change, great for you?

> arch, nix, docker, flatpak, snaps etc

Lumping all of these into the same category doesn't even make sense, other from a "they're all things invented after 2000 and I don't wanna learn anything new" perspective.

> The latter will all "win" on their merits at some point and become fat and lazy themselves as they evolve and grow

Can you back up your nihilism?

> those companies prefer reliable, predictable and boring.

They prefer what they're used to, even if it's neither reliable nor predictable due to having too many moving parts (that don't even make sense any more), and are only boring in the sense that you know exactly how wasteful and brittle they're gonna be and how much effort you will waste interacting with them.

And even then, their preferences don't stop most "boring big enterprise" companies from using newer technologies, just look at docker. Your condescending false dichtomy is entirely pointless.

I suspect it only looks like overkill in the context of the simplest possible example.

Once you start adding things like pre-inst, post-inst scripts, binary redirections with update-alternatives, I suspect the complications start to pay off.

it's 3 files, 1 file that really just specified deb format version and 2 tar files. data.tar.gz (extracted out to file system) and control.tar.gz (a set of metadata file, including installation scripts).

In practice, how would one make it simpler?

i'd note it's simplicity (and the fact that ipkg/ipk format was based on it), was what enabled me to help the people who "jailbroke" (really just install homebrew apps) the palm pre: https://www.engadget.com/2009-06-22-pre-apps-successfully-in...

I was able to assist people that had the device without even having one as the format (to me at least) was very simple.

With regards to the other front-page story about attracting new Debian Developers - https://news.ycombinator.com/item?id=41599327 -

This article is great for understanding the deb format. Lots of tools can make deb archives e.g. https://fpm.readthedocs.io/ . However making a binary package for users is a quite different thing to making a deb suitable for inclusion in Debian itself. A debian source package is a tuple of three .dsc + .tar.gz + .debian.tar.gz files.

I thought a relevant comment on that story by o11c [0] was particularly insightful based on my experience with Debian packaging. The format is probably optimised for long term maintenance or running up against inherent complexity or something (works great as a user) but the process of setting up a package did have a weirdly high-friction feel. The last time I tried to package something for a Debian system it felt a lot more like constructing a fork of the project than distributing something that already existed.

[0] https://news.ycombinator.com/item?id=41638352

Isn't it time to use GNU Guix package manager ? https://guix.gnu.org/

weird timing! made my first .deb tonight...

I appreciate first principles, but i just used dkpg-deb --build

do you have a blog? i've got a thing id like to package for deb but the docs are a bit complex and intimidating

this one's pretty easy to start: https://www.internalpointers.com/post/build-binary-deb-packa...

thanks!

Not OP but I have started with https://github.com/FooBarWidget/debian-packaging-for-the-mod... - it's 4 years old but the basics are still valid. Then I would recommend the debhelper documentation. I maintain a few binary deb packages for my own repo myself. For this I would recommend reprepro.

thanks!

First try updating an existing source package or adding a patch to one. Hell, just download one with `apt-get source` and just read it— look at the files that make up the source package. Then download a few more. Maybe try and build some. Change a couple files before building and see what happens.

Once you have a feel for that stuff, use debhelper to create a new package 'from scratch' (from an upstream source tarball).

This 'tutorial'¹ presents basically the same info as the Debian packaging docs² but it feels more easily digestible to me because of how it's chopped up into slides.

Some of the manpages are also good. The debhelper manpage³ refers you to other small docs, notably dh_make⁴, which is probably your best starting point for creating a package from scratch.

1: https://www.debian.org/doc/manuals/packaging-tutorial/packag...

2: https://www.debian.org/doc/manuals/debmake-doc/ch05.en.html#...

3: https://manpages.debian.org/bullseye/debhelper/debhelper.7.e...

4: https://manpages.debian.org/testing/dh-make/dh_make.1.en.htm...

thanks!

Reply on news.ycombinator.com