x86 Bare Metal Examples: Minimal operating systems, learn x86 system programming | Hacker News Reader

x86 Bare Metal Examples: Minimal operating systems, learn x86 system programming

github.com

These are good examples and this might be slightly off topic but I wish that there were more resources on how modern x86 platforms actually work instead of using these legacy interfaces that are either emulated in SMM with IO port trapping or relegated to requiring chipset hacks to keep them working (such as VGA device hacks in the PCI spec or well known chipset legacy IO ports decoding to the LPC bridge).

While it is essentially impossible to truly understand every aspect of todays platforms I think that one can build a lot of intuition about how things work by studying how transaction routing is handled in the platform, e.g. how the uncore and chipset routes TLPs (whether it's raw PCIe or packets essentially wrapped by other interconnect protocols like IF/HyperTransport/DMI/QPI/etc) through the system with various base address registers and decoding windows. This gives way to understanding how modern MSI interrupts and DMA works as well. While I understand that truly going into things like PCI configuration, ACPI, UEFI, chipset interfaces on bus 0, modern booting, DMA and interrupt remapping, DRAM, register locking and security, security processors, power management, various types of runtime firmware, etc is a a many-years-long exercise I think that the core ideas don't need to be spread around all over the place for one to get familiarity with the ideas and patterns that underpin our PCs.

I think that a general conceptual overview on which one can hang more detailed knowledge acquired by reading specs is needed with the amount of complexity that exists today. There are some really good resources that I've collected over the years but there is still something missing. Maybe I or someone else will get around to it some day.

"legacy" is the reason the PC still remains relatively open despite corporate threats, so don't dismiss it lightly.

https://news.ycombinator.com/item?id=15730113

I'm not sure why you have legacy in quotes, that is what it is. Modern platforms do not act like a PC AT and there is a ton of cruft to keep those interfaces working which is set up and facilitated by firmware (not so open, at least TianoCore exists regardless of your thoughts on UEFI). Modern x86 systems programming is so much more than these interfaces. I think that it would be beneficial for others to understand how things are actually working on modern platforms.

Regarding your linked post:

For corporate users legacy methods of booting /are/ a security threat. Firmware attacks are a real threat for enterprises. Having IBVs all hand roll things like the S3 boot script or communication with SMM (not using standardized com buffers) is not wise. Secure boot combined with modern security features like using DRTM to remove firmware from the trusted computing base are important and should not be discounted because it's not an 40 year old interface. The PC platform is historically extremely insecure and there is a ton of work still to be done to effectively provide the primitives needed to secure the platform against determined attackers.

If you want to have a USB stack in SMM owning your USB controller to emulate port 60h with IO SMI traps that's fine but it's not a reason to hold the platform back.

The insecurity is a feature, not a bug.

Usually double-quotes is just pointing out it's as the other person said, not calling it into question which is usually single-quotes.

I collected some more developed learning OS project at https://github.com/cirosantilli/x86-bare-metal-examples/tree... BTW.

Read about x86-64. There are a few tutorials out there, and most focus on modern systems with the "big four" (UEFI, ACPI, XHCI, PCIE).

I am talking about the platform more than the ISA. Some interesting resources:

https://sites.google.com/site/pinczakko/system-address-map-i...

https://www.youtube.com/playlist?list=PLBTQvUDSl81dTG_5Uk2my...

http://xillybus.com/tutorials/pci-express-tlp-pcie-primer-tu...

https://www.amazon.com/Quick-Boot-Pete-Dice/dp/1501515381/

https://opensecuritytraining.info/IntroBIOS.html

https://acpica.org/sites/acpica/files/asl_tutorial_v20190625...

And of course the MindShare books.

Could you recommend a tutorial?

I don't understand why people like x86 bare metal programming so much!!! XD Just kidding. Related:

- https://news.ycombinator.com/item?id=19428700

- https://news.ycombinator.com/item?id=18531393

- https://news.ycombinator.com/item?id=13773219

It might have to do with all the recent discussions about Windows 11, mandatory TPM, and "secure" boot restrictions; to show that, yes, people do try to write their own OSs, and that such things would be basically impossible if the megacorps had their way.

What's weird about Windows 11 is they are integrating Android apps. In some ways it's getting more open and, with the TPM requirement, in some ways it's getting more closed. Android apps run on Linux with AnBox, though I haven't tried it. It seems like a much more viable world for alternative operating systems when most applications are written for Android.

Nowadays, it's better to use x86_64 only. The 32 bit x86 has many drawbacks. The 64 bit is much cleaner and better in many many ways.

  -no segmentation, flat address space
  -better interrupt handling
  -better system call
  -more registers
  etc.

Also we found that Qemu or other Virtualization technology may cover up some of the hardware issues. Eg. Beside, that every hardware device (keyboard, usb, display, disk etc.) will have some simplified behavior, if you emulate an ARM cpu on x86_64 host, than the cache coherency will totally messed up on real hardware.

Yes, absolutely correct about possible QEMU simplifications. One option would be to use gem5 which can simulate caches/memory more accurately (sometimes), but it is also way slower as a result, some indications at: https://github.com/cirosantilli/linux-kernel-module-cheat/tr...

"Read this before getting started." I enjoyed the jab at Pooh.

I'm also looking forward to digging into this. It looks really neat and I have been interested in baremetal programming recently. Hopefully this coming weekend will be free enough to indulge!

Appropriate avatar

Because even a Paramount Leader needs a little programming side-project on slow weekend.

Great examples, well done!

Now do multicore!!

Thankfully, I regained my sanity before going that far. The "a second CPU started" example at: https://github.com/cirosantilli/x86-bare-metal-examples/tree... though. I think some of the projects under https://github.com/cirosantilli/x86-bare-metal-examples/tree... did it though.

Take a look at https://pdos.csail.mit.edu/6.828/2019/xv6.html

Reply on news.ycombinator.com