Consider using Digital Mars C compiler | Hacker News Reader

Consider using Digital Mars C compiler

virtualbox.org

85 pointsyuhong43 comments

I am probably going to get downvoted for this, but let me try. I am assuming, this is related to VirtualBox using the OpenWatcom compiler [0] for BIOS because it is the only reasonable way to handle x86 segmentation from C.

So this ticket is basically asking the VirtualBox maintainers to take a reliably working part of the product and to port it to a different compiler, retest everything, fix any bugs caused by differences between compilers and update the build process as well. So as an end user of VirtualBox, I would either notice no difference at all (BIOS isn't a performance bottleneck really), or will get a regression due to bugs.

With all due respect to the free software community, I would say switching the compiler used in a stable product without an apparent improvement noticeable by the users would not serve their best interests. It is also a bit strange to see it not in a form of "hey, we really care about free software, so we ported this part to use a free compiler, here's the performance comparison, here are the unit tests, here are the build instructions", but rather in asking someone else to spend considerable effort on implementing this.

[0] https://www.virtualbox.org/ticket/12011

Yea, this is not about performance. This is about license problems. For code like BIOS, pretty much any good 16-bit C compiler should work.

> So as an end user of VirtualBox, I would either notice no difference at all (BIOS isn't a performance bottleneck really), or will get a regression due to bugs.

Yeah, but we're already getting VirtualBox for free, so we can't only take into account the benefits of the end-users ; if the developers find it more convenient to use this new compiler, so be it.

> using the OpenWatcom compiler [0] for BIOS because it is the only reasonable way to handle x86 segmentation from C.

Can you elaborate?

It is not impossible to use gcc to generate 16-bit code, but it lacks support for things like near and far pointers.

>It is also a bit strange to see it not in a form of "hey, we really care about free software, so we ported this part to use a free compiler, here's the performance comparison, here are the unit tests, here are the build instructions", but rather in asking someone else to spend considerable effort on implementing this.

I seem to see this shift in ideology from People.

> because it is the only reasonable way to handle x86 segmentation from C.

That's nonsense, both GCC and Clang handle this perfectly well. It's how we handle compiling 16 bit option ROMs in qemu:

https://git.qemu.org/?p=qemu.git;a=blob;f=pc-bios/optionrom/...

This is the thing I don't get. qemu / SeaBIOS seem to build fine with a normal compiler like gcc or clang, as far as I know, and they're in Debian main. Why is VirtualBox different? What user-visible feature does its BIOS have that qemu doesn't - does it support booting OSes that qemu doesn't, or something?

Can VirtualBox just switch to SeaBIOS?

Notice the special routines required for reading and writing from segments. A true 16-bit DOS compiler handles this for you.

I realized I had neglected to make the source to the runtime library available. Fixed that:

https://github.com/DigitalMars/dmc/tree/master/src

Everything in it that is copyrighted Symantec or Digital Mars is now Boost licensed. In particular, stuff that is copyrighted by Microsoft is NOT Boost licensed and cannot be redistributed.

This is going to sound like an stupid question but it could be just lack of marketting or lack of documentation but I'm reading through the website and it only mentions supports for Win32, Win16, DOS and DOS32, is this correct? I'm curious to know how is that going to work for VirtualBox given that I use it on my Mac. Is the ticket suggesting they switch from VisualC++ compiler to DMC?

It would be for things like the BIOS you use to boot your VirtualBox VMs which has to be 16-bit real mode code. Hopefully the compiler itself runs on Linux already (though this is only needed for building the code).

I don't know anything about VirtualBox. But DMC can generate 16 bit x86 code.

The ticket is suggesting that for building the BIOS that runs inside the VirtualBox image, they switch from OpenWatcom (another C compiler for DOS) to Digital Mars. VirtualBox emulates an IBM-compatible PC, and the traditional boot protocol for IBM-compatible PCs involves a BIOS that sets up the machine and provides a rudimentary runtime library for things like screen output or disk access. This BIOS is written for 16-bit x86, same as / binary-compatible with DOS, so any good compiler that can produce DOS object files can be made to work here too. (Most bootloaders / kernels will quickly switch to 32- or 64-bit mode, but they expect to be started in 16-bit mode and expect to make calls to the BIOS to load their own code.)

This ticket is not about changing the compiler that builds VirtualBox itself.

I noticed that the link to the 32-bit DOS extender that DMC uses is dead. Do you or anyone know of a legal way to obtain it?

Just send me your email.

I agree! https://www.digitalmars.com/download/freecompiler.html

This is almost as bad as the drive-by "rewrite it in rust" type comments some open source projects get.

Why not just use gcc or clang?

Problem is that x86 segmentation support is lacking, which is important for things like the BIOS.

For certain ways of writing the BIOS - you can have the BIOS immediately enter 32/64-bit flat addressing mode (as if you were a 32/64-bit kernel) and then drop back to 16-bit mode when you're done if you need to do a legacy MBR boot. I think this is how coreboot/libreboot/linuxboot/etc. and Tianocore/EDK2 work.

Can you use Tianocore as your BIOS for VirtualBox the way that you can use it (in the form of OVMF) for qemu?

How is SeaBIOS built?

And even writing some 16-bit code using gcc is not impossible too. The problem is that it lacks support for things like near and far pointers.

Without near and far pointers, it is hopeless to write non-trivial programs for 16 bit DOS.

And even for things like BIOS they can be useful, and that is what I am most interested in.

Sure. My argument is that unless you're interested in retrocomputing (which is a cool field, but not the field VirtualBox is in) you shouldn't be writing non-trivial 16-bit programs at all any more - VirtualBox doesn't even support emulating any 16-bit-only CPUs. You should be writing 32-bit programs with a tiny bit of 16-bit code on each end (either in assembly or in trivial C) to switch from 16-bit to 32-bit mode and back. Since you only need the BIOS for a very small amount of time to support a bootloader loading itself, the slowness of switching modes at each interrupt will hardly be your limiting factor (especially given that you're in a VM).

This has the advantage that you don't need any new compiler to build your program, neither OpenWatcom or Digital Mars. (Nothing against the Digital Mars compiler! I just think that it isn't the right engineering direction for VirtualBox to depend on retrocomputing.)

Though it is worth mention that the runtime code that implements BIOS services (INT 10h, 11h, 12h, 13h, etc...) for the most part can't switch to protected mode without breaking compatibility.

SeaBIOS is doing a reasonable job at that

but there is also the VGA BIOS.

Do you even need a VGA BIOS in an emulator? It's an emulator - just write to segment B800 or wherever and have the host-side process just watch that page of memory.

Again, qemu works fine without a retrocomputing compiler. Is VirtualBox doing something that prevents it from taking whatever approach qemu uses? (qemu even has the cool thing where "-curses" will cause it to copy segment B800 as text to your host-side terminal emulator and avoid rendering graphics at all.)

What I am talking about is things like this: https://mail.coreboot.org/pipermail/seabios/2015-April/00902...

You're in an emulator - you don't have to switch to protected mode within the emulation, you can instead exit the VM and do the work outside the emulation. The caller within the emulation won't notice unless it's trying to single-step the BIOS code (and why would they do that?).

Edit: unless the code within the emulator is trying to run the BIOS within an emulator of its own. Turtles all the way down...

>Edit: unless the code within the emulator is trying to run the BIOS within an emulator of its own. Turtles all the way down...

Common with the VGA BIOS BTW.

Even most UEFI/BIOS are compiled by MSVC.and MASM. Why to ignore OVMF?

They don't use x86 segmentation and normal gcc should work fine.

Context: The Open Watcom license has been a problem for years.

Could you please expand more

I had an issue about the problems: https://github.com/open-watcom/open-watcom-v2/issues/271

So there's a theoretical problem, and a solution available.

Why should VirtualBox take the pain now rather than waiting and seeing if the theoretical issue every materializes? I mean, if someone does the work and tests it, then sure, VirtualBox should adopt it. But why is this their problem to solve now?

There is a practical problem that VirtualBox is in Debian contrib and not main because it cannot be built with tools in Debian main. (Whether this is a practical problem that matters to anyone is a different question, but it's more than theoretical.)

I don't think DMC is in Debian main yet, is it?

Probably not yet, but the license means that it can be.

So that is also just a theoretical idea at this point.

Yea, they just open sourced it, so it might take a while for it to be packaged for Debian. I don't even know if it is ported to Linux yet at this point.

Reply on news.ycombinator.com