Fast Debian package builds using sbuild, mmdebstrap and apt-cacher-ng | Hacker News Reader

Fast Debian package builds using sbuild, mmdebstrap and apt-cacher-ng

stephan.lachnit.xyz

60 pointsJNRowe18 comments

A few years ago I needed to add a new arch into Debian, it's a painful process, there are lots of arcane packages used to truly build and bootstrap the debian and they're all cross-dependent on each other. I wish there is a way like "linuxfromscratch" with clear steps to build Debian from source from scratch.

Yes you can bootstrap a Debian root filesystem for a different arch, but that's after someone already did the work before you can do that, which again, is really complex and I have there are better ways to do it now(e.g. adding a new CPU family to Debian)

A lot of older Debian documentation has rotted away which is very sad for retro hobby stuff

I wanted to rebuild Debian 2.2 against GCC 3.3 with its better support for SMP and SSE support for an old Pentium 3 workstation I've been playing with. Back then that kind of optimization could yield impressive performance benefits

Unfortunately there's just no documentation for how to rebuild those ancient versions of Debian. The oldest I could find targeted the much newer Debian 4.0 "Etch" which came out in the mid 2000's

Gentoo still supports i486 architecture and it's totally ready to tweak build flags.

Gentoo also has a ton of historical rot, something users have been trying to solve for a while https://bugs.gentoo.org/834712

Myself and some other folks recently wrote some new docs on how to add new arches to Debian. With multi-arch, cross-compilers in Debian and rebootstrap, it has gotten a lot less painful. There are still a ridiculous number of packages that need porting to every new arches though. Some of the needed changes should be moved into toolchains in my opinion.

https://wiki.debian.org/PortsDocs/New

Might not be exactly what you're after:

https://github.com/scottwilliambeasley/debian-from-scratch

Which architecture were you working on adding?

I've actually used apt-cacher-ng, or at least I spent two years trying. Sadly, it's a buggy pile of poop that constantly segfaults and has serious data corruption bugs. It corrupts packages. Like, all you gotta do is download a file from the internet, and it messes that up by corrupting it.

All of these bugs are super well-known, written about, and documented.

I never did find a good Deb package cache manager. I simply wrote a script that distributes common packages between my LAN systems as I update them in a serial matter. It works but I wish there was something better. Everything I've tried is either sub-Alpha bugware and incomprehensibly complicated.

How about squid or another straight up http cache?

It's easier to compile the linux kernel from scratch than it is to build a debian package from scratch.

It's mind-blowing how complicated it is; at the end of the day it's just an archive. I hate having to do it.

I store files in an appropiate strcture in my repo in a "src" folder, eg "src/usr/bin/myprog", "src/etc/cron.daily/myprog.cron" etc

Then I have a "src/DEBIAN" folder with a control file and maybe things like conffiles, postinst, etc.

Control file looks like

  Package: mypackage
  Version: 1.0.__datestamp__
  Architecture: all
  Maintainer: My Name <my.name@email.com>
  Depends: perl, libcgi-pm-perl, libtimedate-perl, apache2
  Section: web
  Priority: optional
  Installed-Size: 1000
  Description: My fun package which does good stuff
   Do wonderful things like existing

sed replaces the __datestamp__ line with yyyy.mm.dd.hh.mm.ss, then

  fakeroot dpkg-deb --build ./src

builds a deb.

(I have a larger script which does things like checking out from git, tagging, etc)

Actually it is not that hard:

  - default: https://vaporup.github.io/books/the-pragmatic-sysadmin/debian-packaging/
  -   quick: https://nfpm.goreleaser.com/usage/

Read the official documentation instead of random blogs.

Official documentation is a problem. It is aimed at Debian maintainers building a fully policy compliant package that could be accepted to the official repository. You don't need any of that crap for making a .deb package. It puts off people from creating Debian packages for their own programs or private use.

Interesting, thanks. I'll give it a read.

When is Debian going to enable zstd for packages already? It takes forever.

The bug looks closed recently:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=892664#49

Oh, that's good. Is it actually used now in packages in Debian repos though?

Reply on news.ycombinator.com