Compiling OCaml directly to a new cloud operating system
anil.recoil.org
anil.recoil.org
during my weekends when I work on node.ocaml ( http://github.com/mathgladiator/node.ocaml ), I occasionally think "gosh, If I could just get rid of that pesky OS, then I would have the most perfect server ever!".
Under normal kernels, the standard OCaml garbage collector cannot guarantee that its address space is contiguous in virtual memory and maintains a page table to track the allocated heap regions.
They couldn't find ~1 GB of contiguous address space (out of 128 TB available) under Linux, so they threw out Linux completely?
Each Mirage instance runs as on a single CPU core, and depends on the hypervisor to divide up a physical host into several single-core VMs.
GIL getting you down? Just define away parallelism and let the programmer handle it!
x86-64 does not have segmentation, and Xen protects its own memory using page-level checks and runs both the guest kernel and userspace in ring 3. This makes system calls and page table manipulation relatively slow, a problem which Mirage avoids by not context-switching in ring 3.
That processor has hardware virtualization acceleration for a reason. Working around performance quirks in obsolete hypervisors doesn't sound like a good use of time IMO.
In tight allocation loops, the page table lookup can take around 15 % of CPU time, an over-head which disappears in Mirage
Functional languages like OCaml allocate and deallocate memory much more often than traditional languages like FORTRAN. So while you are perhaps correct in some situations, this approach would seem more optimal for functional programming.
Yes, the principles described here could easily be applied to a full Linux kernel. In fact, one of the hacks on my TODO list is to statically link a Mirage application against a Linux kernel (without a userspace) to run them on the bare metal.
The point of defining a single address space and the smallest possible C runtime is to start from the other end: rather than stripping away 14 million lines of C code, I preferred writing a few thousand lines and jumping straight into my runtime. It is an awful lot easier to experiment with something like http://github.com/avsm/mirage/tree/master/runtime/xen/kernel... the full Linux kernel by quite a long way. Note that the current tree isn't finished yet; I'm pulling out dietlibc entirely at the moment, so the final kernel binaries float around the ~200KB mark for a typical webserver. Then, also consider hypervisor-only features like live migration or PV suspend/resume that can be further optimised heavily and more easily in a minios instead of Linux. Or that Mirage is single-vCPU and event-driven only (no interrupts), and it starts to look quite different from Linux.
The Mirage IO library is also very portable; an application which uses only TCP/UDP (instead of the lower level Ethernet) will compile on Linux/*BSD using select/epoll/kqueue sockets to be "just" a high performance webserver. The cool thing is that by using these APIs, we can also build applications that compile to small specialised operating systems, while developing them as usual on UNIX.
For a final entertaining hack, they are also portable enough to run directly in the browser as Javascript applications thanks to Jake Donham's ocamljs project. We're still integrating the Websocket code in to make this properly finished, but it's a fine side experiment into portability :-)
In general I realize that y'all are looking for rationalizations for an exokernel-style design, but IMO they need to be fundamental issues and not bugs.
In erlang, compiling erlang from within erlang is a normal way of doing things, since things are broken down into tiny erlang processes.
As another poster on this thread put it (being sarcastic I think, but actually spot on):
GIL getting you down? Just define away parallelism and let the programmer handle it!
We are indeed building multiple parallelisation strategies on top of Mirage, which work across cores and hosts seamlessly. But first things first, and getting the efficient serial version out is top of the list right now...
Though, I have always wondered why an idea of a free distributed operating system was never implemented. With the old HeliOS, all you needed to do is add it to the serial links and processes would migrate to it as needed. And when a machine was turned off, those processes just migrated elewhere.