An Inside Look at MS-DOS – The design decisions behind the popular OS (1983)
patersontech.com
patersontech.com
One thing that I always thought was interesting was all the stuff that gets packed in the first 100h bytes before a .COM file's entry point, called the Program Segment Prefix (PSP). It is why you could simply execute a RET (return) and have it drop back to DOS again. The first bytes of the PSP are a literal INT 20h which is the Drop To DOS interrupt, and a RET with nothing left on the stack will return you to that zero offset, and then Drop to DOS.
Probably a bit hacky, but really clever.
I wrote my own DOS clone in Assembler, that I called PoverDOS because it was surely going to make me about that much money, and managed to make a boot sector that was far more functional than the one that comes standard with MS and PC DOS. The main thing it did was self-diagnostics to determine if the system had been infected by a boot sector virus. That was quite a feat, but became a moot point once 64 bit computing came about, because the boot sequence also gained a whole lot more space to work in.
Don't bring it up with me at a party, 'cos all those DOS intricacies are still among my favourite subjects, and 97% of the population probably would not be nearly as enthusiastic. DOS is still the party killer it always was.
Ironically I learned how the boot sector worked and basic ASM purely by using DEBUG.EXE. My computer got infected by Stoned virus, and I was pretty excited about it! This was back when DOS came with a green binder with actual documentation. It was a year or so of that before I discovered TASM and then MASM assemblers, which of course are a LOT more human-friendly.
The earlier versions of DOS were entirely written in ASM, and they were super-tight. But later versions were all rewritten in high level languages. The stuff in the leak is all the bloated high-level language versions.
This article was originally published right before the issuance of MS-DOS 2.0. That was not even near the '90's yet.
By the time this was captured in the year 2000, the MSDOS.SYS file had already become almost a dummy file of few bytes in Win9x.
The DOS OS itself with only the native commands is basically contained within the IO.SYS, MSDOS.SYS, and COMMAND.COM files, originally written in that order starting at the first working sector of a formatted floppy disk so it would boot.
On a hard drive the 3 files would have to be written starting at the first working formatted sector of an actively flagged primary partition, and a proper master boot record needs to be on sector 0 (along with a valid partition table) if any boot files on that HDD are to be recognized by BIOS to begin with. Otherwise it would not boot.
IOW once the volume boots you can do things like copy files and list directories, but if you want to edit files or format other volumes you needed more than just the 3 OS files. You needed the additional OS apps like EDIT.COM and FORMAT.COM on your floppy so they would be there when you needed them after you booted.
Once booted, the SYS command could be used to properly write the OS to a target volume, so you needed SYS.COM for that.
And by W9x, the three DOS OS files would no longer need to be right there at the start of the volume (as if they were the first things copied to the volume), and DOS FDISK would handle HDDs of hundreds of GB even though each FAT32 partition was only supported up to 32GB.
The undocumented FDISK /MBR command executed from an A: or B: (floppy) prompt would simply write a fresh bootable DOS master boot record to sector 0 of C: (what should usually be considered) your only HDD on the mainboard. Without disturbing the partition table there.
The SYS command also replaces the Volume Boot Record (bootsector) of the target volume with a fresh DOS bootsector as it freshly (over)writes IO.SYS, MSDOS.SYS, and COMMAND.COM into the file system.
So you want to make a zeroed HDD boot, first boot to a DOS floppy (USB stick will do):
Run FDISK and create a primary partition (bigger than 512MB) smaller than 32GB on the fixed (HDD) disk. Make the partition Active.
(Alternatively you could use some other way to make a suitable partition, like Linux or a modern Windows version but the resulting "geometry" of the disk layout may not be the same or fully recognizable by DOS, which can affect the bootability. Sometimes with Linux or Windows, the older the version the better. Outcomes can also be surprisingly interesting when certain sectors of the HDD are not exactly zeroed beforehand. AMD also not quite the same as Intel, nor old Seagate identical to WD.)
Now you've got a partition table on sector 0 with one primary partition marked "active" so the BIOS will boot to it.
Run FDISK /MBR and it silently writes an MBR to sector 0 so the BIOS will REALLY boot to it.
(You may need to manually copy a working MBR to the first 440 bytes of sector 0 if the FDISK /MBR command does not do the trick. The MBR is supposed to be generic for DOS and a Linux version like MBR.bin from the Syslinux project can be copied bytewise with binary tools. Newer Windows will also upgrade the MBR but it will still boot DOS (or Linux) afterward from a submenu.)
All other sectors are still unchanged.
Run FORMAT on the partition and I would always suggest labeling it as to HDD/Partition# when done. 11 characters max.
Now you've got something other than zeros in places other than sector 0 of your HDD, but still no data actually located on the formatted volume yet.
Run SYS on the formatted volume and it properly writes the 3 critical DOS OS files to the volume.
Power down, remove the floppy or USB device.
The HDD should then boot DOS according to mainboard BIOS priority.
Add W9x to the same FAT32 volume and you still get a DOS boot entry on your Microsoft boot menu for when you want plain DOS.
Install WXP to a different formatted partition (even FAT32) and your old W9x/DOS volume still can boot when desired as a submenu of the NT5 NTloader that replaces the DOS bootloader on that active partition.
Then with Win10 (non-UEFI setup) on a third (NTFS) partition you can have an NTLDR submenu within the replacement NT6 Bootmgr and boot any of the above.
Note the volume you installed the new Windows version to does not become active itself nor contain any boot files. By default additional boot entries are made to the established active partition. Even logical partitions containing OS's can be booted this way, according to OS version, from one of the few primary partitions you are allowed to flag as active.
BIOS only. Too bad with foolish-type UEFI or GPT, no DOS for you.
Also remember with bare-metal DOS/W9x and audio/MIDI (no USB), no latency for you either.
Even on dog-slow PC's.
Things like that.
This overwrote the generic (DOS/Windows) MBR on sector 0 with a Grub-specific MBR which hard-linked to more Grub-specific code stored on otherwise unused sectors located in between sector 0 and the first sector of the (not necessarily formatted) first partition, expected to be at least 31 sectors available for Grub to install to. And for Grub to have those sectors to itself (not true if other apps treated it as scratch space outside the file system).
You could link Grub to other sectors if you wanted to but this was where the default location was.
You could also always have a dedicated boot partition but this did not become mainstream until Windows 7 and it was with BIOS booting. That approach has now been continued into UEFI with the dedicated EFI folder on the hidden ESP partition.
So Grub took over your PC and that's how it was going to boot. Most of the time in Grub1 you had to carefully manually edit the grub.conf file to get your old untouched Windows partition on the boot menu so your dual-boot layout was ideally functional.
This didn't always happen during a user's first Linux experience so the HDD was basically stuck in "Linux only" until they figured it out. If the urgent need to boot the old Windows came up, reboot to the emergency boot floppy containing FDISK.EXE, then run FDISK /MBR and everything would be back to "Windows only" after that.
With Grub2 it kind of got worse since grub.cfg came along in place of grub.conf, and grub.cfg is intended to be autogenerated and not manually edited. The recommended recommendation is to never manually edit the grub.cfg file. I recommend not always doing what's recommended if you want to really get what you want. But don't fail to back up your work in case it autodisappears. In the long run you could really get into Grub2 and make it autogenerate your dreams.
Either way it was much better when installing Linux to a type 83 partition to leave the generic DOS/Windows MBR in position at sector 0, and instead direct Grub to be installed to the same partition (that partition's bootsector) that Linux was destined for. This made the (EXTx-formatted) type 83 Linux partition completely bootable on its own. But only if that was a primary partition to begin with and was flagged active with a binary 80 in its partition table entry in sector 0. And at the time the Windows partition would instead have a 00 flag, therefore not being active and was not the primary to boot that time.
You would have to manually assign which primary partition to flag active so you could multiboot, but it is accomplished that way without even a menu to configure. This was best done manually with a binary disk editor although some menu-interface shells arose such as Bootmagic which handled self-contained bootable partitions especially well. Boot magic also usually installed to "unused" sectors on the HDD outside any file system.
Further, the functional grub bootsector for the partition it is installed to can be copied to a binary file (named something like bootsect.lin) on an active FAT or NTFS partition where an NT install had often autosaved a copy of a FAT DOS bootsector as bootsect.dos. (Before it was replaced with an NT bootsector).
BOOT.INI on the active Windows bootup partition could then be edited to include a boot entry invoking bootsect.lin and the NT bootloader would jump to the target type 83 Linux partition and carry on from there.
So you could install and boot Linux ideally to an extra partition on a Windows PC without disturbing the Windows in any way. Just adding the Linux option to the default Windows boot menu.
When a small separate FAT32 boot partition began to appear before the main NTFS volume on OEM Win7 PC's, that's when Syslinux began to really shine. Because Syslinux is made to boot Linux from an active FAT partition similar to how Isolinux boots Linux from CDROM. Syslinux has its own kind of bootsector and relies on a syslinux folder and/or syslinux.cfg file so for a while there it was a better choice than Grub. You just had to use the same version of syslinux that the distro was isolinux'ed with, and rename the isolinux folder & isolinux.cfg file to syslinux & syslinux.cfg, with judicious manual editing.
Syslinux is stll superior for BIOS booting but for UEFI Grub2 is what seems to work most predictably now.
Under UEFI Grub2 acts about the same way always booting from the dedicated EFI folder on the hidden FAT32 ESP boot partition. It simply overwrites \EFI\boot\bootx64.efi with a distro-specific version from the distro that you are booted to when you update grub. For instance if you install Ubuntu to a Linux-formatted GPT volume and properly direct Grub to install in the EFI folder it makes a new bootfolder \EFI\ubuntu in the process and the new \EFI\boot\bootx64.efi no longer seeks \EFI\Microsoft, it seeks \EFI\ubuntu instead (where it finds Ubuntu's grub.cfg). But it leaves \EFI\Microsoft untouched, and each additional distro adds its own like \EFI\fedora and \EFI\debian, and they all coexist wonderfully.
Don't let different mainboard UEFI implementations fool you, the firmware itself is supposed to recognize different boot options as they arise and you can get a firmware boot entry (available from a separate usually disabled mainboard UEFI bootmenu) for each installed OS, which all may not be functional choices after a while. But \EFI\Microsoft\bootmgfw.efi can be like a hardcoded firmware boot entry as a failsafe alternative (Windows boot manager) to an expected \EFI\boot\bootx64.efi. Either of which \EFI\boot\bootx64.efi or \EFI\Microsoft\bootmgfw.efi are further alternatively executable from a UEFI shell. Whether the UEFI shell is built into the mainboard firmware, or if not you have to provide shellx64.efi on a pre-boot (usually FAT32) file device.
Whichever distro you are booted to will be the one whose grub.cfg is reautogenerated when you update grub. You are expecting the grub update process to autodetect all installed OS's including Windows and place entries for them on a boot menu in grub.cfg which appears routinely every time you restart the PC. Some distros do this better than others.
So it's a good idea to back up the Windows version of bootx64.efi for possible restoration to \EFI\boot before allowing Grub to overwrite it with a Linux version. Then again you might want to back up your Linux versions of bootx64.efi as they come along too.
Pretty much the modern substitute for an FDISK /MBR, although the BCDBOOT command (sometimes requiring the /uefi option) overwrites \EFI\boot\bootx64.efi with a fresh Windows version, while it examines your \EFI\Microsoft\boot\BCD for any corrections it wants to make.
Just to bring things more up-to-date, it's not 1999 any more.
I documented it to death, so it is super easy to read even though it is ASM code.
AIUI that design was in CP/M already, and retained for compatibility.
Thats likely a strong understatement.
https://fd.lod.bz/rbil/interrup/dos_kernel/2152.html
That allowed TSRs to do clever things.
I didn't use CP/M before DOS, but these days I know all about the PSP via writing code for CP/M, in Z80. Kinda impressive how long all these backwards compatibility things last.
I seem to recall 'one guy' had this hand-curated master list of interrupts. Might've been this list; perhaps another.
I was obsessed with TSR programs back then. The notion that you can run a background process (in exchange for gobbling up all-valuable conventional memory) thanks to clever hacking really captured my eye back then.
I still remember the DWIM (Do what I mean) interrupt they used as the demonstration of how each interrupt was documented.
> https://fd.lod.bz/rbil/interrup/dos_kernel/2152.html
> That allowed TSRs to do clever things.
I think they were also used (=manipulated manually) by viruses, in order to prevent DOS from reporting the reduction in memory, and the user from noticing the virus.
Boot sector viruses couldn't do anything like that because they load and run before DOS enters the picture. So that was one of the things my boot sector looked for. File infectors had a number of tricks at their disposal, one of which you just mentioned. Other methods are still similar, just different means to that end.
Although it seems to me that Microsoft already had enough talented engineers in 1981 that they probably could have written a CP/M clone from scratch? But maybe the timeline they promised IBM was too tight, in which case Paterson / SCP was indeed the perfect unwitting savior.
(When these events took place I was wearing diapers, so it’s all third-hand information. The great thing about HN is that there’s probably someone around who was in the room for half of it, and then somebody else who witnessed the other side.)
Submitters: "Please submit the original source. If a post reports on something found on another site, submit the latter." https://news.ycombinator.com/newsguidelines.html
(It's fine to post archive.org links if there's really no other way, but please search for another way first—there are quite a few editions of this article on the web, which you can find by googling sentences from the text—and try to find the canonical source if possible!)
It's a great submission otherwise! and appears never to have been discussed on HN before, which is an unusual find.
The author of this article is the guy who invented the original DOS, which was acquired by Microsoft for peanuts.
If Seattle Computer Products had sold their product to IBM directly under the same terms they would be as rich as Bill Gates today.
And oh the time I have spent to write menu entries in autoexec.bat and config.sys, figuring everything out from floppy disks containing things like cdrom drivers. I wasn't much older than 10 years old back then and was so happy when someone gave me an update: MS-DOS 6.22.