https://www.linuxfromscratch.org/
I failed tremendously, but I had a lot of incorrect ideas about what makes an operating system go. The learning experience was worth it, though.
The go-to guide for anyone that wants to build their own operating system has always been the OSDev community wiki and forum: https://wiki.osdev.org/Main_Page
It's just that Linux is kind of usable out of the box with just the kernel and /bin/sh. But for other OSes this isn't true, the kernel can expose a vastly different interface than the actual os which users interact with.
I actually came into this thread thinking, "damn, I have no idea specifically WHAT I would do/not do if I made an OS, or where to even start" not that I could at all. But I mean feature-wise, etc. Like, it's obviously a ton of apis that you call to invoke things at the system level but I have no idea what to do at the system level. Maybe you just communicate with APIs over there for everything on the motherboard. I know Nvidia has NVAPI I've used that a bunch through their SDK, not more low level. And you start with probably a super baby kernel..
There are also different styles of OS, like windows with the registry which I have broken and fixed thousands of times, and then nix which is pretty much file based. I don't know what else would be better than either of these, I hate the registry thing but maybe it has some huge benefit I haven't figured out.
When I look at C I see heiroglypics. Maybe me learning rust is helping with that. I do really need to learn it for security sake and (live)patching, etc. I write go and ruby so I don't touch memory much.
I'm the same about filesystems. I know a bunch of them and hve done tons of performance testing for SANs etc but couldn't tell you a thing about the lower level, making one, stuff.
edit: lol I guess a lot of this is installing drivers too
So, what exactly happens here - you need to write a kernel in some super fast low level language like C. I THINK The kernel that you write is probably talking to, for instance an Intel/AMD CPU, Motherboard, GPU, etc API to access the hardware.
Is this close/wrong?
If I made a completely new kernel how do I boot it? Does the kernel have its own API that the BIOS calls? Does the BIOS have anything to do with the kernel but instead something starts Windows and that loads the kernel? I think the kernel mounts/becomes aware of all of the hardware (apis?) potentially by the motherboard telling it what's on it. Does the BIOS just start any C app you point it at and they're just C apps that start the kernel?
Maybe it goes through the motherboard for most of it.. Maybe it accesses every piece of hardware directly without an API somehow?
This may be stupid questions but hopefully they make sense.. And thanks if you or anyone has input!
edit: There's a Boot processing of linux Wiki [1] thats pretty good
> After being loaded into RAM, bootloader (also called first-stage bootloader or primary bootloader) will execute to load the second-stage bootloader[2] (also called secondary bootloader).[6] The second-stage bootloader will load the kernel image into memory, decompress and initialize it then pass control to this kernel image.[2] Second-stage bootloader also performs several operation on the system such as system hardware check, mounting the root device, loading the necessary kernel modules,...[2] Finally, the very first user-space process (init process) starts, and other high-level system initializations are performed (which involve with startup scripts).[2]
> For each of these stages and components, there are different variations and approaches; for example, GRUB, coreboot or Das U-Boot can be used as bootloaders (historical examples are LILO, SYSLINUX or Loadlin), while the startup scripts can be either traditional init-style, or the system configuration can be performed through modern alternatives such as systemd or Upstart.
> The intermediate stage loader (stage1.5, usually core.img) is loaded and executed by the stage1 loader. The second-stage loader (stage2, the /boot/grub/ files) is loaded by the stage1.5 and displays the GRUB startup menu that allows the user to choose an operating system or examine and edit startup parameters. After a menu entry is chosen and optional parameters are given, GRUB loads the linux kernel into memory and passes control to it. GRUB 2 is also capable of chain-loading of another bootloader. In UEFI systems, the stage1 and stage1.5 usually are the same UEFI application file (such as grubx64.efi for x64 UEFI systems).
[1] - https://en.wikipedia.org/wiki/Booting_process_of_Linux#:~:te....
[2] - https://www.gnu.org/software/grub/manual/grub/html_node/Imag...
[3] - Potentially how systemd starts installing the kernel https://github.com/ivandavidov/systemd-boot/blob/master/proj...
[4] - This looks like it's a TPM authentication shim that sits between the EFI and Kernel - https://github.com/ivandavidov/systemd-boot/blob/5521e37e77a...
[5] - Ok this is super helpful, a project on writing a minimal kernel - https://www.codeproject.com/Articles/1225196/Create-Your-Own...
[6] - And wow the GNU bootloader spec is rough - https://www.gnu.org/software/grub/manual/multiboot/multiboot...
I know fundamentally it's all the same in regular computers, but I am unfamiliar with the structure of it.
There are a couple of us true full-stack developers but you're very right that we're not common. Programming in low-level languages allow you to think about how, precisely, your memory will be laid out, but on the flip side, they require that you think about memory layouts. If you enjoy thinking about such things, you get used to planning for cache optimization and so on, and moving to python requires actively turning that part of your brain off and taking a „que cera cera" attitude to any performance consideration lower level than big-O notation.
Would be interesting if someone with knowledge (e.g. David Chisnall) could help clarify whether this seeming of mine is seemly.
The reality is there have always been other languages[1], and not all of them codified the evil stream-of-bytes paradigm. We've been writing OSes in Ada since the 80s. Pascal variants were somewhat common back in the day (e.g. Charles River PERC, CDC Cyber NOS). Apple was on the Pascal train for a long time. Unisys Clearpath is still written (and sold) in their highly extended Algol variant (ESPOL)[2]. I've used OSes written in Modula-2 & Modula-3. There's a number of folks that are trying FreePascal as an OS language.
Of course, there was a healthy LispOS population at one brief, shining point in time (sigh...Genera). Tektronix and a few others had Smalltalk machines.
IBM still writes a lot of low-level stuff in PL/I dialects across it's proprietary product lines (i Series, z Series). Intel wrote it's first ones (e.g. iRMX, ISIS) in a PL/I dialect (PL/M). And CP/M is there too.
Then you can get to the really niche OS writing languages: JOVIAL, anyone?
[1] The source code for some of these legacy OSes is out there. Studying it might be instructive. [2] Clearpath is the current incarnation of MCP from the Burroughs Large System mainframes (e.g. B5000, B6500, etc.). Stack based and very interesting; now runs as a VM on top of Xeon servers. Not only can you still buy Clearpath, you can download a hobbyist version to try out locally: https://www.app5.unisys.com/library/gmmail/emails/documents/...