Building binary deb packages: a practical guide
internalpointers.com
internalpointers.com
Debian and Ubuntu automatically resolve shared library dependencies are build time through dpkg-shlibs and friends. Debian's maintainer manual is likely the most detailed guide: https://www.debian.org/doc/manuals/maint-guide/start.en.html
However, it's a bit hard to grok, so I'm going to point people at the guide that got linked in the comments before I got here: https://saveriomiroddi.github.io/Learn-to-prepare-PPA-packag...
It's completely fine to package them in a deb. You especially should know that debs are really simple and work quite well for that. You place the binary along with the needed resource files (configuration for example) in the relative directory structure, add a minimal amount of meta information, and call one command to create the .deb. Easy and fast, and the result is very reliable, it will work everywhere.
And then you have the official way of abstraction over abstraction, "explained" with completely unreadable documentation, needing a dozen helper tools and environment variables, guided by documentation that fails to explain even just the basics of what it is you'd actually do if you followed that documentation.
No, the simple way shown by the submitted article is exactly correct.
Start with that. If it works, then you maybe take the next step later and set up a PPA, assuming the deb is not only for yourself.
For someone new to this, the article's 5-steps are certainly not "practical".
What is the problem with distributing a tarball if you're sending binaries?
If you're making debs, you still need to make a Debian repo and sign it, and then you still have the problem your dependencies are fucked up or won't easily work across versions.
What are you trying to solve that a tarball or rsync can't?
End users can install a deb with one click without having to know about where to put the files, or how to create a shortcut in menu.
These types of debs cause problems with system upgrades and I groan whenever I see something that uses checkinstall or similar; there are add-on deb sites that basically do that, and you get a mess in the resulting system. A lot of this is why snap got made.
I'll admit that the Debian package format could be better in this regard, but the packaging format is primarily for use by dpkg-buildpackage and building the distro.
Disclaimer: this is my own views and not those of either the Debian or Ubuntu projects.
Official Debian development, being FOSS oriented, proceeds from Debian source packages instead, and one never creates a binary package from scratch as described in the article. Instead, we use `dpkg-buildpackage` and other similar tools. The creation of the source package is the hard part, having to follow all the aspects of Debian policy, including technical matters around quality.
I'm too embarrassed to even link to the sources that I created.
https://mentors.debian.net/intro-maintainers
Debian packaging documentation is unfortunately a swamp, since people continue to write new guides instead of improving existing ones.
example: I got a tool that allow from any Windows/macOS/Linux to package a runtime+script to any Windows/macOS/Linux
the Windows .exe can not be compiled from sources under Linux, so this particular runtime can never be distributed on Debian official?
https://packages.debian.org/search?keywords=mingw
Probably the right thing for you to do is switch to WinGet on Windows and the Apple Store for macOS.
"portable" and "crossplatform" ("multiplatform") are different degrees of software.
In contrast, I managed to submit software to Homebrew after two weeks of macOS, and to Nixpkgs within two days of running NixOS.
The tradeoff you get with other distributions being quick, is less quality assurance. Also, Debian still has a policy of working on 10 architectures, and this slows things down significantly sometimes. The other distributions you mentioned don't have this constraint.
I dare say, if Ubuntu is any indication, Pacman packages consistently feel higher-quality than deb packages. Unlike on Ubuntu, on Arch, you can actually update your packages without breaking something...
The crux of the problem is that all current official Debian packaging methods supported by Debian are sufficiently arcane and obtusely documented to the point that it's a high barrier to entry for people to do it right.
Personally, I've been doing RPM and Debian packaging for over a decade now, and it boggles the mind how much more complicated doing _correct_ Debian packaging is for essentially the same that happens on an RPM distribution like Fedora or openSUSE.
One of the reasons that I started using debbuild[1] for building Debian packages years ago was that I felt like I could get something that closely resembled a policy-compliant Debian package without impossible amounts of guesswork. Combined with macros adapted from Fedora and other distros[2] as an interface for Debian packaging policies, I can package things rather easily for Debian/Ubuntu using the much more straightforward guidelines from Fedora as a base.
As it turns out, it doesn't take that much effort to make packages that comply with Fedora/openSUSE Packaging Guidelines and Debian Policy with this tool, and it has drastically simplified my ability to maintain software across distributions in a way that still cleanly integrates with the distribution platform.
I wish that debbuild would be an officially supported mechanism in Debian, but it's probably unlikely without a Debian Developer advocating for it (and I suspect no one there would, as they probably like the existing system...).
[1]: https://github.com/debbuild/debbuild
[2]: https://github.com/debbuild/debbuild-macros
(Disclaimer: I am now the current developer of debbuild and debbuild-macros, though I wasn't when I first started using it)
Simply throwing a bunch of files into a package or a container is very quick.
Making an official Debian package is not supposed to be quick. DDs thoroughly review and test the software they are packaging. While packaging I often chase missing licensing information, find plenty of bugs, write systemd unit/init files and sandboxing, write manpages, functional tests, and sometimes find serious vulnerabilities.
I worked on various packaging and deploying systems, and most other distributions don't come anywhere close to this level of scrutiny.
I should also add that becoming a DD requires years of commitment and proven track record of work.
The next time someone complains that making a container is very easy in comparison they should ask themselves: how much can I trust a software source with a very low entry bar?
Can you clarify what do you exactly mean with this?
Also... look at that guide. Why is it so difficult and obtuse? Why do I have to run so many odd tools, including apparently modifying some file in /etc and running other stuff as root? In contrast, this was my first contribution to Nixpkgs: https://github.com/NixOS/nixpkgs/commit/0b8efd8724e59964bad0... Easily as reproducible as anything Debian, and apart from installing Nix it didn't require any funkyness in the build environment. I tested the definition by running 'nix-build . -A oclgrind' in my checkout of the Nixpkgs repository. I managed this after running NixOS for two days.
This is the same software that I also contributed to Homebrew: https://github.com/Homebrew/homebrew-core/commit/1268459f761... Again, a simple declarative-ish package description (written in Ruby, which is not my favourite, but it's better than the weird Makefile/shell amalgamations that are used by DPKG and RPM).
Both Nix and Homebrew have serious limitations compared to Debian (Homebrew in particular is aggressively limiting its scope to gain simplicity and in particularly doesn't care as much about reproducibility). But as a user, I must admit I have not missed what Debian provides that these do not. Whenever I tried to package stuff for Debian, I felt that I was paying a tax for unclear gain. It's a shame, because I like Debian for over a decade, and I am strongly sympathetic to the idea of the project in principle.
I agree that Debian packaging is difficult/odd/arguably obtuse; I guess it's a legacy matter. Your phrasing can be interpreted as if the guide itself is obtuse, it's not clear :)
> including apparently modifying some file in /etc and running other stuff as root?
It's important to read and understand it fully. Anything you do it as root is required only to run a preview build locally. This is an optional step, so no, one doesn't need to perform admin configurations/operations if they don't want to, although performing a local build is a desirable step.
> In contrast, this was my first contribution to Nixpkgs
This is not a proper comparison, as you're comparing an entirely manual procedure versus using a packaging tool.
A proper (closer) comparison is probably against the tool that the guide references: https://github.com/saveriomiroddi/ppa_packaging, where you just declare all the metadata, and invoke a script that takes care of everything.
All in all, I'm personally puzzled about why there aren't tools around to simplify/automate Debian packaging. I've started the small tool mentioned above (`ppa_packaging`), but I'll likely not continue working on it, as my experience with niche projects is that, while useful in their (narrow) domain, they're very resource-intensive to maintain.
I mean Debian packaging is a terrible pain - but I have to have empathy with the fact that it's a technical system and more importantly an online community that's been doing it's thing since 1993.
The fact is that with such a long-running system there's more than one way to do everything.
The way we discuss/learn technology has also partly changed. Most of the documentation for Debian is extensive manuals - there's "not enough" intro text, and then "too much" detailed documentation
And then there's the "cost of change". Lots of Debian developers have been packaging using their particular versions of tools for a long time and are happy with them.
In many ways Debian works because the people working on it care about quality and the tooling is second order to the efforts they make.
(someone with the applicable knowledge please fork and continue it)
The memcached people have provided a systemd service unit since 2011, for 8 years at the time that M. Bernat wrote that article. One quite clearly does not use the -d and -P options, nor Type=forking. memcached is Type=simple per its authors.
Indeed, Type=forking in a systemd service unit is a good indicator that its author does not understand service management. It is invariably wrong because almost no programs actually speak the forking readiness protocol. (They fork for rather different reasons, which do not in fact adhere to the requirements of the forking readiness protocol.) See it, and you know to treat pronouncements about how daemonization works from that source with a great deal of suspicion.
One such here is "memcached is started with the -d flag and will fork when it is ready to accept requests". In fact, memcached with -d forks before pretty much all of its initialization, and long before it is ready to accept requests. It hasn't even opened its listening sockets, or opened any data files or started any worker threads, at that point. Client services started because "memcached is ready" can fail to even connect to it. It might indeed yet fail whilst initializing, and never become ready.
Really, memcached should have been runnable under inetd as a "wait" service from the start, which it unfortunately is not. That could have been adapted into a program that speaks the LISTEN_FDS protocol; and it would have a memcached.socket unit, with Accept=No and some ListenStream and ListenDatagram settings, and a Type=simple memcached.service unit. Then clients would have been able to reliably connect to the socket as soon as the service was started, and queue up until the service begins to accept() them when it becomes ready. Early socket opening in action.
* https://github.com/memcached/memcached/blob/c460f0e13a35dcec...
* http://jdebp.uk./FGA/unix-daemon-readiness-protocol-problems...
I've written a book about Debian packaging that spends most of its page count on building Debian packages with debhelper/dh, the approach that most (or all?) official packages use.
Here's a link for the HN crowd to get it for free: https://leanpub.com/debian/c/pm
Happy to answer any questions as good as I can :-)
</plug>
You will read about debian/rules file, but then open one from any example project and only see a single rule calling "dh". Turns out debhelper is there doing things for you. On to read about debhelper. But now sometimes you'll see some rules file that uses the target "override_dh_autoreconf", which was not explained, not even hinted, in the previous docs. On to read more.
Did you think you got everything? Not so fast!
You'll document yourself about debian/control files and see that using some variable-like names are common in lots of projects. For example this is used a lot:
Depends: ${misc:Depends}, ${shlibs:Depends}
On with the same issue. For some reason (which new users reading the docs shouldn't really have to care), "shlibs:Depends" seems to be used all over the place, but it is not mentioned, hinted, or even suggested in a mere "see also" paragraph in the docs about debian/control. If you are thorough enough you'll end up seeing a quick mention in a different section (8. Shared Libraries), and turns out the actual definition of this variable has to be checked out from "dpkg-shlibdeps", an independent package.
Now how to actually build a package?
In the old days, the debian/rules file contained all the steps, but all is now helpfully abstracted by simply calling debhelper (dh). But calling debian/rules itself is abstracted also by dpkg-buildpackage. Calling dpkg-buildpackage is itself covered by debuild. And debuild is possibly covered by calling git-buildpackage. There is a crazy pile of tools calling tools calling other tools, that is nothing but extremely confusing for newcomers.
For the 99% of times you just want a local package for local installation, dpkg-buildpackage is what you want. But getting to that conclusion is not something you get pointed to from a brief introduction from an official manual. You get to that conclusion after either studying all tools and putting order to all scattered knowledge in your head, or after blindly following some random blog of forum post (which tend to be correct because thankfully all this tooling doesn't change much over time)
EDIT to stress that all information is there, but the official docs lack a good enough effort of linking or including everything in such a way that it follows a logical learning process for newcomers.
In the past it felt like DSCs uploaded to the FTP queues were the equivalent of committing code on a software project. Are they all in one git repository now, that can be tracked outside the project?
Nothing beats being able to read the code for learning how a system actually works.
apt-get install maint-guide ...and then read:
file:///usr/share/developers-reference/index.html file:///usr/share/doc/maint-guide/html/index.en.html
Also look at existing packages and talk to DDs. Avoid random wikis.
I worked on many deployment systems and reading these guides made me a better engineer overall.
The problem I saw from parent and other posts was that there’s two sets of information you need to build a package:
(1) the sources and debian patches, where the last public release of these is available from packages.debian.org
(2) knowing the right command to run to build the package.
In the old days you could guarantee everything would build via dpkg-buildpackage though you’d need to know to add -r fakeroot.
It sounded like nowadays it’s more complicated than that?
The problem is not that, but the way to get there, and how the information is structured and presented, especially if you start from having no idea, or if you want to build your debian/ files from a blank slate and being able to read in a centralized resource about definitions of every line you add (which I think should be possible to do, for any system in general).