Easylkb: Easy Linux Kernel Builder | Hacker News Reader

Easylkb: Easy Linux Kernel Builder

tmpout.sh

134 pointssjuut28 comments

Mind. Officially. Blown. Sometimes the coincidental timeliness of things popping up on Hacker News is just weird. u ROCK deepseagirl!

I've recently wanted to start building Linux kernels and play with device drivers again. Wanted to build and then execute on top of QEMU. Downloaded, configured, and built Linux kernel 6.6 following recipes from arch and a few other places. Doing a reasonable .config was a painful process (picking what to include etc.) The kernel build took several hours. The resulting image refused to boot under Virtual Machine Manager, to say nothing of using gdb etc. to interactively debug the kernel.

So, I figured I was looking at a solid weekend of googling, stack-overflowing, swearing, reading kernel Documentation files etc. and finally getting everything set up and working happily.

NOT! The easylkb script did the entire thing completely automatically, culminating in a login prompt on a running virtual machine!! The Linux kernel .config was one of the standard ones I believe, but I hadn't yet found it. The complete kernel build took on the order of several minutes instead of several hours.

There were a couple of minor issues I had with easylkb, which I will share as comments on github.

In the kernel source directory do:

    make help | less

and the various config targets are listed; to use the default configuration (for the current host architecture) do:

    make defconfig

To edit it further use one of the edit options such as menuconfig:

   make menuconfig

All of these create/operate on ./.config in the source-code 'root' directory.

This and much more is covered in

https://kernelnewbies.org/KernelBuild

Another useful tip is to grab an existing config, eg. the one for your current system found under /boot, rename it .config and then do 'make olddefconfig' (followed by building the kernel in the usual way).

Distro kernels are normally quite "full fat" so this may tend to build a lot, but it's a good way to get a kernel which closely matches what you're typically running, both in terms of device coverage and compilation options / hardening.

Greatly appreciate the suggestions and pointers! (Not sure how HN orders "reply's", but this is in response to other replys to my initial comment.)

I found the NixOS wiki[1] to have a fairly decent guide on how to hack on a kernel. The process there isn't nearly as concise as it is here though :)

[1]: https://nixos.wiki/wiki/Linux_kernel

I personally wen't this route last year and eventually got things working (networking etc. OK with only the kernel). My company then took a direction in which each compute node has its custom kernel, so that we can control the initial stages of boot better (initrd is dynamically downloaded from intranet, given that the network drivers are baked-in) to manage heterogeneous nodes with different "roles" (GPUs, add-on devices, etc.).

Something I have been surprised about is how much functionality exists in the Linux kernel and how much more reliable the modules are compared to many user-space tools (e.g. race-conditions).

I've wanted to do some Linux kernel development for a while now, but I've never mustered the will to go through the process.

This script may just be what I needed to start.

I just want to learn. Unfortunately those days with simple source commentary long gone. Hence whilst we have this good tutorial about setup … but how you start to learn more about.

Considering and size and complexity of the Linux kernel, I don't think any amount of source code comments would help you much, unless you already have a strong understanding of the underlying subsystem you want to hack on.

"Understanding the Linux kernel", and "Linux device drivers" are pretty good, pragmatic books IMHO to give you a solid understanding of most useful subsystem.

As for this kernel building script, I guess it could be handy, but unless you're hacking very frequently I never found the actual build of the kernel to be the more cumbersome part.

the kernel and device drivers are both a bit out of date unfortunately, hope LF can found new revisions of them.

How relevant are these books today? They were both written in 2005.

A bit out of date, but not to the point of being useless. I still give them to juniors and interns that want to learn more about kernel work. There's also a much more recent (2022) book called Linux Device Driver Development published by packt. In general, these sorts of books are mainly useful for "what not to do", as general architecture stuff is usually well documented by source code examples and in-kernel docs.

Thanks!

> unless you're hacking very frequently I never found the actual build of the kernel to be the more cumbersome part

Doesn't kernel use incremental builds? All well-designed C projects that I know of compile .c files into .o files and link them, so a single file modification only requires a single recompilation/relinking.

It does use incremental builds so long as you don't reset everything.

I'm not sure that a lot of people who don't work with make realize that it's smart enough to only rebuild what has changed, and that it's not just a command runner!

If you want to get a good introduction, I recommend the bootlin kernel course. All course material is available on their website.

https://bootlin.com/training/kernel/

Thank you! This looks really good

The Linux Foundation has similar offerings but 3x more expensive, less time, and online only https://training.linuxfoundation.org/linux-kernel-developmen...

Ooooh this is very cute.

I'm toying with building a little OS using the Linux kernel and an all-Go userspace, and one of the goals is to make the entire system as easy to cross-compile (both host OS and host/guest CPU arch) as possible. Linux being non-trivial to compile (let alone cross-compile) has been so far quite a nuissance, so I'll definitely be having a closer look at this.

cross compilation equally non-trivial. set CROSS_COMPILE= and ARCH= and making the appropriate changes to .config (either set ARCH and `make defconfig`, or modify the relevant parts in `make menuconfig`). then build as usual

The idea there sounds a lot like https://gokrazy.org/, which builds a minimal go userland, wrapping one or more user provided go applications, and bundles in a linux kernel.

Targets mostly at single board computers, and I think it downloads pre-built kernels (and bootloaters if needed), rather than trying to build them directly, since getting a working cross compilation toolchain set up and plumbed into the kernel compilation process is still a pain.

I've personally only used yocto/open-embedded for that which does nicely handle building the cross-compilation toolchain, kernel image, and modules. But it is kinda overkill for that task, being designed to build a whole userland too.

2nding gokrazy as a very nice embedded target for pi's.

Yep, both gokrazy and u-root serve as inspirations/references (there's a lot of tricky stuff like configuring the network or the framebuffer). Using a kernel from Debian while figuring out my options - glad there seem to be so many.

The ultimate goal is to make it very straightforward to change any part of the system source and try it out in action, so the only things you need installed to get started are Go and Qemu. Further down the road I'd love to extend these principles to hacking the kernel - I'd probably like a simpler kernel, maybe OpenBSD or NetBSD, but I'm less familiar with those.

You might enjoy looking at one of my projects, a toy Linux distro compilable from a single Makefile. https://github.com/snacsnoc/snacklinux

It uses musl and builds using a musl cross compiler, working x86 and arm64 builds. You could easily build a toolchain for your desired arch and make a simple distro, adding in any user space tools.

This caught my attention because I'm obsessing about storage performance (again) and have included a kernel build in my benchmarks. I found and followed the instructions at https://wiki.debian.org/BuildADebianKernelPackage which were trivially simple.

As an aside, I was astonished that the build took less than 100s. My recollection is that this took about an hour on on my previous system (I7-4770K, 32GB RAM and SATA SSD formatted with ZFS.) New system is Ryzen 7 7700X, 32GB RAM and NVME SSD. I expected a performance boost, but not that much. I'm still wondering what I did wrong/differently.

I didn’t have a very hard time kernels, but building the gnu toolchain is an absolute disaster

It indeed is a journey for the most patient of us. Or, at least, was about 8 years ago, when I needed to build gcc for the Cortex M3 target. I guess not much has changed since then, and one still has a great opportunity to learn what mpfr, gmp and the rest of scary words mean, before either giving up or pushing through that crap. It's like eating super hot peppers: first one suffers, then enjoys the knowledge that they are a tough one. And maybe an endorphin rush, if they have suffered intensely enough.

Definitely useful knowledge, but it sucks that’s what the world runs on in my opinion. It really does just need to be built from the ground up again

Debian had make-kpkg for a while.

Reply on news.ycombinator.com