Ask HN: What Comes After John Lions' Unix Commentary? | Hacker News Reader

Ask HN: What Comes After John Lions' Unix Commentary?

Any suggestions from the group here what is a good follow up to the classic John Lions’ commentary on Unix? Do I spring into the tanenbaum books on OS design? Looking for opinions. Feel free to ramble. Thank you!

View on HN

It's not at all "new" anymore (published 1999), but if you enjoy source-code commentary then check out ISBN 978-1576104699 - "Linux Core Kernel Commentary: Guide to Insider's Knowledge on the Core Kernel of the Linux Code"

Very similar approach to Lions. There's a second volume that covers the networking stack as well.

It depends a bit on your objectives - do you want to build your own OS, learn more about the theory and algorithms used in operating systems, or do you want to know details about commercial OSes?

For a hands-on view on building operating systems, I can recommend taking a look at xv6 [1] as the next step. This is a modern reimplementation of a subset of the 6th edition Unix, running on RISC-V in its current version (an older version targeted x86) [2].

The classical textbooks used in OS courses (Tanenbaum's "Modern Operating Systems", Silberschatz' "Operating System Concepts" and Stallings' "Operating Systems") are more on the theoretical side, whereas Tanenbaum's Minix books are more hands-on. A current, somewhat more hands-on, textbook is "Operating Systems: Three Easy Pieces" by Remzi H. Arpaci-Dusseau and Andrea C. Arpaci-Dusseau. Additional bonus - it's free [3].

For more hands-on resources, there's a lot of material available. osdev.org [4] provides lots of OS development knowledge in wiki form, Stephen Marz' "The Adventures of OS: Making a RISC-V Operating System using Rust" [5] is a hands-on course written as a series of blog posts to build your own OS for RISC-V (but it needs to be updated since RISC-V has changed in some details since the articles were published). I learned about OS theory and practice using Doug Comer's Xinu books [17] and I still think these are very useful.

You can then dig into older Unix(-like) systems, e.g. BSD [6,7] and/or check out more modern approaches such as Microkernels (I can especially recommend L4, starting from Jochen Liedtke's papers [8] and checking out work on seL4 [9]), capability-based systems (CHERI [10] and an old, but good overview book on capability architectures [11]), or - for something a bit more exotic - Plan 9 [12].

If you want to gain an insight into commercial operating systems, there are a number of interesting books on Apple's OS X and iOS [13,14], Windows [15] and even VMS [16].

  [1] https://github.com/mit-pdos/xv6-riscv/

  [2] Russ Cox, Frans Kaashoek, Robert Morris
      xv6: a simple, Unix-like teaching operating system
      https://pdos.csail.mit.edu/6.S081/2020/xv6/book-riscv-rev1.pdf

  [3] https://pages.cs.wisc.edu/~remzi/OSTEP/

  [4] https://wiki.osdev.org/Main_Page

  [5] https://osblog.stephenmarz.com

  [6] Marshall Kirk McKusick , Keith Bostic, Michael J. Karels 
      The Design and Implementation of the 4.4BSD Operating System
      Addison Wesley, ISBN-13: ‎ 978-0201549799

  [7] Marshall Kirk McKusick, George V. Neville-Neil, Robert N.M. Watson
      The Design and Implementation of the FreeBSD Operating System
      Addison Wesley, ISBN-13: 978-0321968975

  [8] https://en.wikipedia.org/wiki/L4_microkernel_family

  [9] https://sel4.systems

  [10] https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/

  [11] Henry M. Levy 
       Capability-Based Computer Systems
       https://homes.cs.washington.edu/~levy/capabook/

  [12] Francisco J Ballesteros
       Notes on the Plan 9 3rd edition Kernel Source
       http://www.r-5.org/files/books/computers/internals/unix/Francisco_Ballesteros-Notes_on_the_Plan_9_Kernel_Source-EN.pdf

  [13] Amit Singh
       Mac OS X Internals: A Systems Approach
       Pearson 2006, ISBN-13: 978-0-321-46795-9

  [14] Jonathan Levin
       *OS Internals (Volume I: User Mode, Volume II:Kernel Mode, Volume III: Security & Insecurity)
       https://newosxbook.com/home.html

  [15] https://learn.microsoft.com/en-us/sysinternals/resources/windows-internals

  [16] VAX/VMS Internals and Data Structures
       http://www.bitsavers.org/pdf/dec/vax/vms/training/EY-00014-DP_VMS_Internals_and_Data_Structures_1984.pdf

  [17] https://xinu.cs.purdue.edu

Thank you. I’m familiar with OSDev however my main gripe with them historically is that it was incredibly disorganized and written by amateurs who weren’t good at doing system design. The information was scattered and had lots of incomplete articles. I’m sure my frustration comes through. Maybe situation has changed today.

I’ve flipped through Windows Internals years ago to find out how their multi-level scheduler works.

I’m interesting in putting together an OS for fun. Don’t even care to blog about it. I have experience with assembly and embedded systems with relatively little to no OS overhead, mostly 8-bit. Reverse engineering isn’t that bad either. Data sheets of processors don’t frighten me, usually. I hook their pins up to oscilloscopes and watch the lines go. I’m looking for an engaging, hands on book written by a competent systems engineer that is paced for programming. Tanenbaum/Minix 3 looks promising. Is this a ground up book with assignments or does he hand you a ready-made OS?

Yeah, osdev is mostly useful for finding out about nitty gritty details (especially for x86).

The Minix book explains details of Minix 3 (file system, memory management etc.) and gives a listing of the Minix kernel, but does not really explain how to write a system from scratch.

Comer's Xinu book is much better suited for this, it uses sort of a literate programming approach mixing code pieces and the related descriptions. However, Comer's books are rather old by now and the most modern architecture covered is MIPS.

If you want to start from the ground up, i.e. setting up a (cross-)compiler toolchain, building a kernel that boots with your bootloader of choice, starting with "Hello world" and then proceeding to syscalls, interrupt handling, cooperative and preemptive multitasking, virtual memory, etc., then Stephen Marz' blog is a better resource.

You could also check out the lecture slides for my operating systems engineering course if you're interested. In this (master level) course, we build a small preemptive multitasking OS with virtual memory for RISC-V from scratch in C: https://multicores.org/ose/

I’ll check these out and I also appreciate you putting your slides online.

I definitely don’t mind RISC stuff, my embedded knowledge is somewhere between ARM and z80 offshoots. I started looking at Minix 3 book (I bought it and forgot I bought it many years ago, it seems) and am reading the source nodding and noodling along. Got caught up reading the interrupt controller data sheet. I remember seeing university courses using Minix as a foundation for building more things on top of it or replacing the default implementation of a piece with something else. Not sure if I can find any of those curriculums and assignments! I’ll have to play around with it. Im not looking for cutting edge data structures, just the general ensemble of putting together a kernel, various management pieces, and some sort of shell.

Plan9

Link to the reading material if you don’t mind?

  What makes Plan9 different:
  https://posixcafe.org/blogs/2024/07/27/0/

  Why Plan 9 by Russ Cox
  https://pdos.csail.mit.edu/~rsc/plan9.html

  History and Motivation
  http://pub.gajendra.net/2016/05/plan9part1

  Plan 9 from Bell Labs
  http://9p.io/sources/plan9/sys/src/

  GitHub
  https://github.com/0intro/plan9

Reply on news.ycombinator.com