Getting into Linux Kernel Development | Hacker News Reader

Getting into Linux Kernel Development

cyphar.com

106 pointsinfosecau29 comments

In all honesty, if you're only a beginning OS hacker, you probably shouldn't be diving into the Linux kernel unless you have a keen specific interest in it. Besides all the complexity, both intrinsic and accidental from its adoption by many heterogenous parties with different interests, it just isn't architecturally interesting.

You want something to broaden your horizons, and that has a much smaller surface. Try Minix 3, Plan 9 (or the 9front fork), DragonFly BSD, Genode, Haiku (BeOS-like), some flavor of illumos, HelenOS or even GNU Hurd.

Besides the easier entry and the higher learning opportunity, you have more of a chance to make truly major contributions. Something like porting your favorite package can get your hands dirty quickly.

Arguably, having less conflict of interest from corporate benefactors makes for a much smoother communication experience, too.

This does not make any sense. Any modern kernel(including the ones you mentioned) is going to be complex. The only way to get started is to dig in. Even if you do not make changes right away, trying to understand the code is a reward in itself.

I also don't get your point about conflict of interest. There is no one corporate sponsor of Linux. And just because you are a newbie, your patch is not looked down upon. There is a great community willing to help if you put in the effort.

A while back I did some fooling around with eCos, and wanted to add a floppy driver. I looked at the Linux one, some from BSD, but settled on the Minix code: it was a lot simpler and easier to understand. I'm sure it was less efficient, and handled far fewer weird corner cases, but it was just a matter of a day or two of hacking to port it to eCos.

It really depends on your goals - if you just want to learn, then Minix is pretty good.

Minix 3 is quite efficient. I would suspect that the complexity comes from the design choices of each OS. Monolithic kernels tend to be more complicated to be maintained than microkernel OSs.

Yeah, illumos doesn't sound like a simple example.

illumos actually has the advantage of a rich set of tooling designed for kernel development (DTrace, mdb, etc.) as well as an ethos of detailed comments -- and it tends to be a bit of a self-selecting community in that it attracts people who value the same. As an example of the adventures of a relative newcomer to illumos, see the excellent writings of Ryan Zezeski, e.g. [1][2][3].

[1] http://zinascii.com/2015/illumos-5498.html

[2] http://zinascii.com/2014/crossed-signals.html

[3] http://zinascii.com/2014/a-posix-queue-implementation.html

Thanks!

20% of contributions to Linux are by people who aren't backed by corporate interest. That's better than most open source projects of even moderate size.

All modern kernels are going to be complicated. Simple kernels aren't efficient or full-featured, that's just how the real world works.

There is no prioritisation of people who are and aren't backed by companies. All patches are judged by merit. And anybody can give a major contribution (just take a look at the massive amount of work going into kdbus which is being worked on by many different people with different interests).

All modern kernels are going to be complicated. Simple kernels aren't efficient or full-featured, that's just how the real world works.

I sense a real bias as to how module boundaries should be architected here. Right off the bat, saying all modern kernels will be complicated (as opposed to complex) and that simple kernels won't be efficient or full-featured, demonstrates you're thinking squarely in terms of monolithic kernels. Are QNX and L4 not simple, small yet efficient kernels?

Complexity is intrinsic, but something being complicated usually implies the use of wrong or incomplete abstractions. Having the right architecture from the beginning can make huge differences, as certain problems either go away or become more manageable as they're shifted into the library or server level, with all the advantages that come from being there.

just take a look at the massive amount of work going into kdbus which is being worked on by many different people with different interests

kdbus is mostly worked on by the core systemd contributors + Greg K-H, which is a handful of people. The fact that it has been routinely criticized by Lutomirski and others speaks for itself.

I'd warn against working on obscure operating systems and kernels unless you were independently wealthy so that this could truly be a hobby. You may invest years of your life into an OS, only to find the job market virtually non-existent. It's possible to switch kernels later in your career (and having worked on another kernel provides valuable perspective), but good luck convincing employers of that (some get it, but they are harder to find).

There are plenty of jobs for Linux and FreeBSD, and I find both kernels very interesting. As for architectural interest, I can see a case being made for Plan 9 or unikernels, but most other kernels are pretty similar.

I don't see the point here. The majority of OS hackers are hobbyists to begin with, and learning should be the top priority in any event. Learning for the sake of being employed as a kernel hacker sounds backwards and an unhealthy approach. You demonstrate your skills and then get hired or apply for a job after the fact.

Besides, it's not like you can't be a Linux sysadmin/programmer in your day job and hack on more novel OS as your hobby.

For example, Al Viro was an ex-Plan 9 hacker who later became a full-time Linux guru.

The thing is that once you've been involved in truly different OS architectures, migrating to a relatively boring Unix monolith like Linux isn't all that hard. Besides, Unix and Linux are so widespread there's little chance you won't be using them in some form throughout.

As for architectural interest, I can see a case being made for Plan 9 or unikernels, but most other kernels are pretty similar.

Sorry, but this is a plain falsehood.

Sorry, I should have been more clear: your point was about architectural differences. I find unikernels and distributed operating systems like plan 9 very architecturally different (for reasons I'm sure you know). I'd probably include microkernels like Minix 3 as well.

But monolithic kernels, like Linux, BSD, and Solaris, being architecturally different? I think if your goal was to explore different kernel architecture, you'd be better off looking at the others, as these are relatively similar. I think you'd even agree!

There is also http://eudyptula-challenge.org/ which might help in familiarising with the codebase in sequential steps.

IMO the issue with the Eudypluta Challenge is that they focus on the wrong part of kernel development (I couldn't get past one of the first few stages because I couldn't figure out what kind of semantics they wanted for the Makefile -- something that isn't a problem for most kernel development work). Kernel development is mostly modification of existing source code, not the creation of new modules (unless you're adding the support for a new driver).

But yes, anything like that does help familiarise you with the semantics of writing kernel code.

I agree. But there are some people(like me) who like a more structured approach, at least initially. I am no kernel developer, so can't comment in that respect with the right technical know-how, but for learning something new, a sequence of challenges is quite good for me to get my feets wet.

And the focus on the damn email attachments. I want to learn to code, not figure out how to use mutt to send attachments that aren't base64 encoded... I can figure that out if/when I actually submit a patch.

As a step 0, people may also be interested in working through Linux From Scratch where you compile your own kernel, set up the file system and create your own Linux distribution.

I didn't see a link to Aleksa's patch set, so here it is for those curious: https://lkml.org/lkml/2015/2/22/204

I'll update the post to include it, but the actual merged patchset is this one: https://lkml.org/lkml/2015/6/9/320. This is the first part which was merged separately in 4.2: https://lkml.org/lkml/2015/6/5/857.

Ah, my bad. So the patch set I linked is something that is still being worked on or just didn't get merged yet?

I was a bit confused as the patch set I linked to were from Feb 2015

That was the first version. There's quite a big difference between that version and the one that got merged (it underwent a complete rewrite because the original version was insanely unsafe -- I'm surprised it even worked on my machine).

This site is really difficult to read on mobile devices (profile photo overlaps first paragraph of text, paragraphs are very narrow due to large padding/margins). A couple of little fixes would make it a much more comfortable read.

Linux kernel development is also really difficult on mobile devices...

Yeah. I should get around to doing that. I'm not a web developer by trade, so I've always felt iffy about doing responsive designs. :/

Feel free to reach out if you need a little help fixing the mobile usability issues - I'm happy to give some advice. I'm not a designer by trade but there's nothing huge to fix there - a few small tweaks to the existing design would make it readable on mobile.

And for new projects, a quick CTRL-Shift-M in Firefox while you're creating your stylesheets would help verify that things you're adding won't break on mobile.

(Sorry if I'm sounding negative, but your post might be really good and some small design tweaks would make it readable on any screen size and help stop the design getting in the way of your content).

I just fixed it by hiding the image and fixing up the padding on screens <1000px. That makes it look fine on my phone screen. I'll take a closer look at it later.

The article links to Linux coding style guide: https://www.kernel.org/doc/Documentation/CodingStyle which has some snarky comments. Is it written by Linus Torvalds?

`git blame Documentation/CodingStyle`

That's not helpful in this case. Most of lines points to 1da177e4, which is:

    commit 1da177e4c3f41524e886b7f1b8a0c1fc7321cac2
    Author: Linus Torvalds <torvalds@ppc970.osdl.org>
    Date:   Sat Apr 16 15:20:36 2005 -0700

    Linux-2.6.12-rc2
    
    Initial git repository build. I'm not bothering with the full history,
    even though we have it.
    ...

Reply on news.ycombinator.com