Linux Kernel Module Programming Guide | Hacker News Reader

Linux Kernel Module Programming Guide

sysprog21.github.io

339 pointsbelter42 comments

Nice, I remember reading the old 2007 version a few years ago, so nice to see it's been updated for modern linux.

Perhaps one of the authors could update the old page to notify/direct users to the new version? Still ranks first when I google for it

Old version: https://tldp.org/LDP/lkmpg/2.6/html/lkmpg.html

Wow, turns out I don't think I've ever read that!

Bit disappointed that this doesn't cover custom parsing of module parameters: always thought that was a bit underused.

Discussion from 3 weeks back: https://news.ycombinator.com/item?id=28036619

This looks really good. :)

Pity it's on GitHub Pages though, so people using IPv6 only can't access it. :(

this appears to be a link to .PDF of the content :

https://github.com/sysprog21/lkmpg/releases/download/latest/... [PDF]

So offer to mirror it then. The D in git’s DVCS stands for “decentralised”.

I’ve heard some dumb complaints in my time but the real beauty of this one is you made it on another site that’s IPv4 only (HN doesn’t yet support IPv6 either).

D in DVCS actually stands for distributed, but your point still stands ;)

Damn it, I knew that as well. That’ll teach me not to post on HN at 1 in the morning!

Thanks for the correction :)

If it were something I was likely to be using a bunch, then I probably would.

Saying that it's a "dumb complaint" to point out that large parts of the world can't access the content is kind of taking the piss though. :(

Saying “large parts of the world can’t access [IPv4]” is massively over exaggerating the situation. Particularly when most of the web is still IPv4 only.

You do actually realise that even users with only an IPv6 address can still get access to IPv4 sites via bridges (the names of which escape me off hand)?

Not the world's biggest fan of GitHub myself but to be fair they're about to support ipv6. They've started putting up AAAA records around other parts of the site if memory serves.

Cool, that's really good news. :)

Do you have real examples of people that actually have zero access to IPv4 sites, or is this just a theoretical concern?

My ISP sometimes has outages of their carrier grade NAT infrastructure. Then the ipv4 is broken, but services that use ipv6 still work.

Quite a few of our (sqlitebrowser.org) users weren't able to access our download server, nor nightly build server, until we made it available over IPv6 in addition to just IPv4.

Seems to be mainly people located in South East Asian countries, though I'd have to go trawling back through log history to get the exact details.

Thanks, that makes sense. Often it feels like that argument gets thrown around disingenuously. It's nice to have specific examples to talk about.

See if your router supports DNS64 (DNS server maps IPv4 addresses in responses to IPv6 addresses under a chosen prefix) and NAT64 (firewall translates IPv6 LAN packets with addresses under that prefix to the original IPv4 packets, and back). I don't know about consumer routers, but it's relatively straightforward to do on pfSense / OPNsense.

It doesn't work for protocols that embed IP addresses, like BitTorrent, since you'd need a protocol-aware proxy for that rather than just an IP firewall, but for something like HTTP there won't be any problem.

Hold up.... People are using IPV6 only networks now? There aren't ipv4 bridges? That's kind of disconcerting. This is a first time hearing of such. While I have no doubt it can be done technically but why can't you get to IPV4 networks?

I’ve always been fascinated by kernel development. Realistically though, what are some examples of useful modules someone could write without several years of practice?

I wrote some low level code to turn an Arduino into a USB audio device and bit banged audio from a couple pins. Basically no experience in hardware stuff. But to be clear, getting good at programming is definitely a year+ kinda thing, of full time focus.

Expanding on this reply, you can think of Kernel Modules as Windows drivers. Most of the time they are meant to be used as an interface on which you can communicate from the OS with an underlying device more easily .

And usually they are paired with a user space tool for even easier interaction. Ideally the driver should expose literally every function and possible action and then the userspace tool has the job of simplifying that down and not supporting very unusual or complicated setups.

You could think of the userspace side as the gui you get with gpu drivers which interacts with a separate kernel space side

How did you end up doing USB with the Arduino? Was it bit banged as well (VUSB)? Is this code available anywhere?

Interfaced with an FTDI chip maybe?

The newer arduino's come with a pretty powerful IC for the usb interface. So I reflashed that using LUFA. To be clear the audio quality was awful, but could certainly hear what was playing. Here is the repo for it. I did it as a self directed class in university. Treat it like I had no idea what I was doing trying to understand lower level programming! https://github.com/ElementalWarrior/ArduinoAudio (This was in C, not part of the Arduino ecosystem with the IDE)

Probably nothing that will get added to the upstream source but you could make infinite file system modules for things. Userspace FUSE would probably make more sense but this would be a good way to get started.

The general vibe I got when looking in to this is that the Linux kernel has more than enough contributors right now so they make no effort to make it beginner friendly or offer an easy on ramp. IIRC Linus said they have trouble simply reviewing all the changes they receive and if you want to help out Linux, your time would be better used on the non kernel projects.

Things like Gnome or Libre Office would be more than happy to receive help.

And testing, I believe the Linux community needs more testers.

It really depends what you mean by making it "beginner friendly".

A lot of work goes into refactoring and cleaning up code, making things cleaner, adding documentation, adding static and dynamic checkers for locks, memory errors, and various other things, making useful common data structures and code into libraries, inspection and debugging tools, reimplementing as much hand-coded assembly as reasonably possible in C, adding a staging area in the tree and moving to a distributed vcs, there are automated CI tests that get kicked off when you post a patch to a mailing list.

In my opinion it is far nicer to develop for, and a lot of code is far nicer to read than it was 20 or even 10 years ago. It's simpler to write some basic hello world filesystem or device driver module because APIs have become a lot cleaner and there are a lot more checks. It is also now far easier let's say to develop a high performance multi-queue block IO subsystem that is mostly lock free and can steer interrupts to specific CPUs based on context and have all the platform specific PCI and interrupt issues and the different memory ordering behaviors of different CPUs all just work. Linux already has one of those though.

Outside of device drivers and new hardware enablement for really simple device types or minor updates to an existing device type, what the kernel is doing now is vastly more complicated. So it is a lot of work to make a significant useful change. This isn't because there is no interest in making it beginner friendly, it's because a lot more of the "easy" things are done.

My source for this comment is the Linux Device Drivers Development book from about 2004 still being the primary source of information on linux development and when asked about an updated version the response was something like "Back when it was written we needed more developers on linux but not anymore". As well as the comment from linus which I can't find anymore where he says other OSS projects need help more than linux.

> My source for this comment is the Linux Device Drivers Development book from about 2004 still being the primary source of information on linux development

The kernel has never had any official development guide or book. Various tutorials and howtos and books come and go along the way. Another good one,

https://www.kernel.org/doc/gorman/pdf/understand.pdf

They don't really scale at all. The device driver book scratched the surface of device drivers even back then. You can still read things like that and get an idea of the concepts which remain pretty similar, but the best way to really learn it beyond hello world stuff is getting into the code. If you want to write a device driver, start with a copy of a different driver of the same class which is well maintained, for example.

> and when asked about an updated version the response was something like "Back when it was written we needed more developers on linux but not anymore".

Not sure of the "we", but Linux kernel developers are in pretty decent demand today and the number of paid contributors seems to be pretty well growing. Certainly a lot since 2004.

> As well as the comment from linus which I can't find anymore where he says other OSS projects need help more than linux.

Yes that has long been true.

I'm not saying you're simply wrong, and I'm sure some people would find great benefit in more or more up to date books on it, but there has been no point where developers just decided that's enough of making Linux more friendly to newbies or have an easier learning curve. Nor has there ever been a great deal of effort to make it easier for newbies mind you. But as I said, people are always trying to clean things up and improve docs and add more seat belts trying to do staging and mentoring and things etc.... but it's just a difficult thing to improve.

My source is only my anecdote of contributing to Linux for ~20 years, so I admit I could well be biased or blind to some of these issues.

I've been wanting to write modules to bring more of Plan 9 to Linux by exposing character devices in /dev for TCP (and probably other things, but that's the big one).

One of the reasons this probably hasn't already happened is because bash specially handles those so they work already. From the man page for bash:

       Bash handles several filenames specially when they are used in redirections, as described in the following table:

              /dev/fd/fd
                     If fd is a valid integer, file descriptor fd is duplicated.
              /dev/stdin
                     File descriptor 0 is duplicated.
              /dev/stdout
                     File descriptor 1 is duplicated.
              /dev/stderr
                     File descriptor 2 is duplicated.
              /dev/tcp/host/port
                     If host is a valid hostname or Internet address, and port is an integer port number or service name, bash attempts to open
                     a TCP connection to the corresponding socket.
              /dev/udp/host/port
                     If host is a valid hostname or Internet address, and port is an integer port number or service name, bash attempts to open
                     a UDP connection to the corresponding socket.

Edit: Not to imply it's not a good idea. I think it is, so much so that the only reason it wasn't done long ago is because often a "good enough for most people" solution can sap the need for a different solution which has other benefits.

I'm aware, but I want that functionality outside of BASH.

It's not so much that kernel development is all that difficult, but 99% of the things you could make that would be useful would be device drivers. And device driver development requires an understanding of digital logic and the domain-specific "hardware APIs" (registers and stuff) exposed by various types of hardware.

So you would basically either have to be from the company that made the hardware, or have the full documentation for it, or be an insanely talented reverse engineer.

Well, the one good example I have from a few years ago was a relatively simple ADC PCI card. I did have documentation from the company. They had a Windows driver, but for Linux, their advice was direct memory writes (which requires root privileges). I didn't want the application to run as root, so I wrote a Linux diver for the card.

You need to have the datasheet.

These are usually a Google search away and the way people develop drivers even in professional settings.

For simple devices (e.g. I2C/SPI sensors) the functionality, number of registers, and procedures are usually very limited and it is not difficult.

For simple devices probably user space is just fine. i.e. https://www.kernel.org/doc/Documentation/spi/spidev Usually only if you need an IRQ or something does it require kernel space.

Another way to learn how the kernel module work is by tracing the kernel code..

    Learn about FTRACE - https://jvns.ca/blog/2017/03/19/getting-started-with-ftrace/

    Use ftrace to trace your favor .ko
       - simple one such as serial port driver. 
       - usb driver
       - usb to serial port driver, 
       - usb camera driver
       - gpu driver (opensource intel/amd)

Look for a random I2C chip that has not been upstreamed yet (a sensor of some kind). You can then hook it up into a Raspberry Pi style board and use a similar device as the starting point for your new driver. It can be hard if you're not knowledgeable, but not a "several years of experience needed" thing.

I've written both I2C and SPI based modules for specialty hardware. They were small and highly specific, but quite useful.

Missing anything about DMAs and IOMMU

Reply on news.ycombinator.com