Hacker News Reader

Comment by jcoffland | Hacker News Reader

I've been using Debian for over 10 years and I still love it as a user. But as a developer, I find it extremely frustrating. I've several times attempted to figure out how to package my open-source projects [1] [2] for Debian but the process is a nightmare. As I understand it, I first have to find someone with appropriate privileges to mentor me. I should be able to just submit a potential package for review.

Then there is a ton of documentation on creating packages but which 300 page guide is the right one to use is unclear. And which set of packaging tools should I use? In the end, the time investment required to get started has kept me from contributing.

[1] https://camotics.org/

[2] https://foldingathome.org/

View on HN

Agreed, I've tried and failed to make Debian packages, and have no idea how to proceed.

In contrast, with Gentoo and their documentation, it was straightforward for me to make my own additional, separate repo, look at other ebuilds to see how packaging works, and have everything just work.

https://github.com/billsix/billsix-portage

I submit ebuilds to Gentoo itself if I think others would benefit.

As far as I can tell, packaging for Arch is straightforward as well.

NixOS was also easy to get started with, I really like it !

Agreed, Gentoo and Nixpkgs both have really helpful people onboarding new packagers.

NixOS (Nixpkgs actually) is indeed incredibly simple to contribute to.

It's a monorepo, and packages are declarative. So for most usecases, its just 5-10 LOC describing the source location, dependencies and build process.

After that, if it builds on your local copy of Nixpkgs, it will build on the same Nixpkgs commit as everything is purely functional. A simple PR is all you need.

For updates, there's super low burden too. Thanks to packages being declarative, everything is quite explicit. So a bot can check for source updates upstream, update your package and rebuild it. All it requires is simple maintainer approval.

Yeah the packaging part of NixOS/Nixpkgs is very pleasant. You write a "config" in the Nix language (example[1] of vorbis-tools, it's <40 LOC), open a PR on Github and it'll be submitted provided you follow the CR feedback. As a user you'd later download the built binary from the NixOS cache so you don't have to build it yourself.

[1]: https://github.com/NixOS/nixpkgs/blob/fce8f26af6ef8209c7d282...

And FreeBSD ports (which, if I remember correctly, inspired a lot of things in the Gentoo packaging system).

Shameless plug: https://github.com/hoffa/debpack

It won't pass all lintian tests due to the ridiculous requirements and useless ceremony "correct" Debian packages need. Still working on it.

If we're plugging tools we use to make Debian packages without dealing with the insanity of Debian tooling, I use debbuild[1] with OBS[2] for the bulk of my packaging at work for Debian/Ubuntu systems.

As it turns out, it doesn't take that much effort to make packages that comply with Fedora/openSUSE Packaging Guidelines and Debian Policy with this tool, and it has drastically simplified my ability to maintain software across distributions in a way that still cleanly integrates with the distribution platform.

This is how the Spacewalk Debian/Ubuntu client packages are built[3][4], among many other things. There's also a few other examples out in the wild[5][6][7].

At least for the stuff I make for myself, I haven't made packages that fully pass lintian, but it's not terribly difficult to do if you want to.

[1]: https://github.com/ascherer/debbuild

[2]: https://openbuildservice.org/

[3]: https://gitlab.com/datto/engineering/spacewalk-debian-client...

[4]: https://build.opensuse.org/project/show/systemsmanagement:sp...

[5]: https://pagure.io/python36-flatpkg-deb

[6]: https://pagure.io/rpmdevtools-deb

[7]: https://build.opensuse.org/package/view_file/Virtualization:...

Another shameless plug:

https://github.com/kakwa/amkecpak

It doesn't exactly simplify the Debian packaging in my case as it leaves the packaging mostly exposed.

But it provides a lot of far more convenient and easy to remember commands (make rpm, make deb, make deb_repo, make rpm_repo, etc) compared to the easy to forget options of rpmbuild/mock and dpkg-buildpackage/cowbuilder.

It also glues everything together from upstream source recovery to building the final repository ready to be rsynced on a public server.

Probably won't help with Debian official packages but I've found success with these materials.

https://linuxconfig.org/easy-way-to-create-a-debian-package-...

And then a quality of life enhancement says push the package building off to FPM.

https://github.com/jordansissel/fpm

Personal Repo

https://www.digitalocean.com/community/tutorials/how-to-use-...

> Community: If a newbie has a bad time, it's a bug.

This is the root of the Debian newbie community experience. FPM seems to get it at least.

> there is a ton of documentation on creating packages but which 300 page guide is the right one to use is unclear

So true. There seems like there are competing packaging approaches, some claiming others are outdated while themselves say the opposite.

I've been willing to package a few of my open-source projects as well for almost a year, and out of frustration, I've ended up building my .deb packages manually and hosting them on my own apt repository. In the meantime, I've published a few packages on PPAs (for Ubuntu) and on AUR (for ArchLinux), and it's been as easy as it could have been.

Given I use Debian exclusively on my servers, it feels like a missed opportunity to me (not that I think my own projects are so much valuable to others, but I'd package a lot more stuff if it was clear how to do it well).

I like the Arch/AUR approach too as you don't even have to host your own repo. With a good AUR helper that system is quite convenient for maintainers as well as users.

The problem with the AUR is its semi-official status takes the pressure off the 'community' binary repositories, while not being up to the same standards - many packages are out of date or don't build, and the complete lack of gatekeeping renders the user vulnerable to malicious packages. Note that "AUR helpers" are not officially condoned by Arch Linux, despite the system being unusable without them - officially you're supposed to examine all the PKGBUILDs of all dependencies manually to make sure they're safe.

I prefer the Void Linux approach - every "template" (equivalent to Arch's PKGBUILD) is kept in a single, monolithic git repository, and the barrier to contribution is kept low; just submit a pull request, and a core developer will review and accept it. This way all packages have binary builds and all packages have had at least nominal review, yet the repository is surprisingly broad. And if you need something that isn't in the repository, adapting an AUR PKGBUILD is trivial.

> Then there is a ton of documentation on creating packages but which 300 page guide is the right one to use is unclear. And which set of packaging tools should I use?

I've run into the same problems, though I've only wanted to build packages for my own use, or company-internal use.

I think I've understood now how packaging is supposed to work, and have written a guide, shameless plug: http://leanpub.com/debian/c/hn (coupon link to get a free copy; feedback would be appreciated).

> feedback would be appreciated

I'm just looking through it, for me looking through the epub version the layout is broken (at least viewing using Adobe Digital Editions), apparently every time there's a yellow block with code.

"1.1 Target Audience and Prerequisites" is the first example but I keep finding more.

Thanks, I'll have to bring that to leanpub's attention, the PDF looks fine.

https://wiki.debian.org/Packaging points to the official documentation.

Thanks for this!

My first upload will soon be 10 years ago. One thing that is definitely noticeable is that while creating a package is still by no means trivial, it has definitely become much, much easier.

10 years ago, you had a handful of very idiosyncratic build helpers to help you manage your package.

Today, effectively it's just one [1], debhelper, and it has become trivial to build packages with it. It's a very powerful framework that can be overridden do do basically anything you like, but usually automagically does the right thing. Gone are the days where you had to write an unwieldy debian/rules file.

3 years ago, you had X version control systems supported: git, svn, mercurial, cvs, and who nows what else. Sounds unreasonable, right? Well, when you rely on volunteer work, you need to live with the fact that volunteers will choose the particular tooling they like to work with.

Today, you have Salsa which is a GitLab instance, and people apparently just learned to deal with it, and the world hasn't come tumbling down.

I absolutely agree with the author that some things are just wrong within Debian. However, I believe these are inherent to the nature of an organization comprised entirely of volunteers; achieving consensus becomes really hard because nobody wants to be told how to spend their free time.

On the other hand, being involved in Debian has also been an incredibly formative experience. I believe Debian does some things right that others still fail at or trail after.

[1] https://anarc.at/blog/2019-02-05-debian-build-systems/

[2] https://salsa.debian.org

The Debian tooling can be really fantastic at time. If the upstream is clean (project with a proper setup.py/Makefile/CMake/Makefile.pl/etc), you only have to declare the meta-data, the build and runtime dependencies and you are set.

But at the same time, I'm not a big fan of other aspects. For example, the fact the packaging is split between at least 3 or 4 files and generally something close to a dozen (at least control, rule, changelog, copyright and maybe .install, .conffiles, .postinst, .preinst, etc) is a bit complex at first specially compared to rpm where everything is centralized into one .spec file.

I'm also not a big fan of how Debian packages handle permission different than root:root. dpkg-statoverride in post install doesn't feel natural, the newcomer while typically use chown, which is wrong. rpm is better in that regard, you explicitly state the permissions in the spec file %files section.

And I'm also not a big fan of Debian renaming upstream archives (<pkg>_<ver>.orig.tar.gz), IMHO, upstream should be taken as it is, including the file name. But that's a minor one and also a very personal opinion.

I've mixed opinions about packages with a lot of scripting to handle a kind of default but slightly customized configuration. I know it can be disabled with "DEBIAN_FRONTEND=noninteractive" but I feel like it's a lot of efforts and an unecessary source of complexity and bugs for a distribution that will primarily be used on servers where such helpers are almost useless specially using tools like puppet/chef/ansible/saltstack.

The tooling also feels a bit like an aggregation of scripts put on top of each. For example, when you want to build in a clean and throwaway environment (aka a chroot), you have pbuilder, cowbuilder, qemubuilder, whalebuilder... CentOS/Fedora has mock. Granted it's less advanced (apart maybe from mockchain) but at least it the obvious choice.

For a lot of things, the Debian packaging feels like Perl: there is 10 ways to do it. But 9 of them are wrong, and you have to go through dozens of pages of policies with no search box (https://www.debian.org/doc/debian-policy/index.html) to figure out the right way. And it's also kind of difficult to follow examples (by downloading source packages to see how it's done) since it's not explicit where each part fits, at least for a newcomer.

I'm curious, in what way do you feel it's less advanced? Mock supports leveraging qemu-user-static for doing foreign architectures as of Mock 1.4.11, which I think was the only major feature that the Debian tools had for a while that Mock lacked.

Is there something else missing?

Mock doesn't have COW (copy on write), cowbuilder by contrast creates a base chroot, then use copy on write to enrich it per build. This speeds-up things considerably.

Also, cowbuilder runs fine with several instance running at the same time, mock is a bit more sketchy in that regard. Basically, you can only build one package at a time. Typically, it's a bit of an issue for my personal scripts: https://github.com/kakwa/amkecpak, in them I can basically do make rpm -j 42 to build 42 packages at the same time, but with mock/mockchain, the chroot creation has some concurrency issues.

In fairness, I used the rather old mock version packaged by Debian, I've not looked at 1.4, it seems that some of these issues are addressed now.

I’m surprised and sad that that’s still the case. I was the creator and maintainer for an open source application some 10+ years ago, and figuring out how to package it properly for Debian was essentially impossible.

It eventually got packaged by someone who had been involved in the Debian community for some time.

I had this exact same problem and also gave up. It was absurd. The amount of friction in contributing to Debian is enormous.

And, at the risk of upsetting people.. it is time to move away from mailing lists, especially for submitting bugs and managing packages.

Debian hasn't used mailing lists for reporting bugs since the 90s: https://en.wikipedia.org/wiki/Debbugs

Debbugs is basically a mailing list.

The original problem with reporting bugs to a mailing list was mails for many bugs were mixed and you couldn't easily get the state of individual bugs. Debbugs, by automatically creating a new mailing list for individual bugs, does solve this problem admirably.

The new problem is that new people hate emails. I think the merit of this problem is debatable.

In the long run, everyone who tolerates email is dead.

But do they die before everything else?

Mailing lists have had threads for decades.

Broken email clients and careless people break those all the time though...

You can subscribe to individual bugs in Debbugs, you can't subscribe to threads. Threads also don't have short ID (bug 123456) and don't have metadata like whether the bug is closed.

My experience trying to update a package was similar. I use Ubuntu so I had the extra friction of using their Launchpad service first, then being told to go-to the Debian lists instead.

It should be easier to make a Guix package and submit it. Then you could run Guix on top of Debian, or use Guix functionality to make a container. Either a tarball or a docker container if that's more your style. https://www.gnu.org/software/guix/blog/2018/tarballs-the-ult...

> I've been using Debian for over 10 years and I still love it as a user.

I've been using it for even longer than that, but I no longer love it. It has declined to being "just OK" in my eyes, which is why I've begun the effort to shift all of my machines to something better.

Why not using CMake/cpack? It can generate 100% valid Debian packages and you don't need to do a C++ project, but you can package whatever you want.

I hadn't heard of cpack - it seems like there's not that many projects using it. Do you have any examples? Do the packages pass lintian checks?

Reply on news.ycombinator.com