Why not port the Linux kernel to Common Lisp? (2009)
stackoverflow.com
stackoverflow.com
Linux took many years to get to the point where it had the stability and performance to be able to compete with commercial operating systems. Users of Linux will not care what language the kernel is written in. They're only going to care whether it will run all of their existing applications without changes, as fast and reliably as Linux does. If the new system doesn't, it will never have any users. (People were willing to put up with the early versions of Linux because it was free and its competitors were not. Any new OS would have to compete with an existing free OS.)
Who would write and maintain it? Linus and most of his followers would stick with the C version, which they know and love, so where are we going to suddenly find the hundreds or thousands of contributors who are both expert Lisp programmers and expert kernel hackers?
I would add: if someone really _did_ do all the work to write a whole modern operating system in Lisp, it would be criminally unimaginative to use the Linux ABI as your interface to the world.
You would have to write your own userspace too for the whole enterprise to be worth bothering with. And then you have basically written SqueakNOS (http://squeaknos.blogspot.ch). :-)
For many of the same reasons people started writing OS kernels in C rather than assembly: Less scut work, wrong code looks wrong more often, and more opportunity for code reuse.
Other reasons include being able to provide a higher-level API, which only kinda happened with the move to C. A Lisp API would enable things like system-level garbage collection and possibly things like a single-address-space OS where the kernel is responsible for fine-grained memory protection and multiprocessing becomes as cheap as multithreading without becoming as unsafe as multithreading.
But that wouldn't be Linux anymore, I suppose; it would be a Linux-compatible system with extended features beyond what the C kernel provides. On the other hand, is Linux 3.11 the same as Linux 1.0 or is it merely mostly-compatible with Linux 1.0?
That's a one man effort to write a Lisp OS for x86 from scratch. He wrote his own compiler, basically with the chain of macroexpansions bottoming out in WITH-INLINE-ASSEMBLER.
I only played with it once. I was struck by how cool it was that I could play with the Lisp REPL from inside the garbage collector (also written in Lisp). This was surprising but then seemed obvious: the GC is simply a Lisp function that's moving objects from old-space into new-space (and then deleting old-space), and it doesn't mind if you wake up and interleave a few more allocations into new-space.
Movitz was current back in 2004 when Pascal Costanza organized the first real european Lisp workshop in Oslo and a whole bunch of Lisp hackers who only knew each other through the 'net got to meet up in person. fun times.
But I wouldn't go for Common Lisp. I'd go for something smaller and cleaner.
1. The ability to manually manage memory instead of relying on a GC.
2. Low-level access to memory. As in, the ability to address an arbitrary location in physical memory
3. Inline assembly and the ability to link with assembly code
There have been operating systems written in Lisp in the past, such as the operating systems for the various Lisp Machines. https://en.wikipedia.org/wiki/Lisp_machine
In other words, the problem is not with Lisp, nor with a lack of support for low-level features, but with tooling. The tool support for writing a kernel in Common Lisp is pretty bad at the moment, and there are not many people trying to improve on that.
http://www.sbcl.org/manual/#Foreign-Function-Interface
You can disable the garbage collector with without-gcing or gc-off. To manually allocate and deallocate you need to go beyond SBCL, at least as far as I know; you can call malloc and free directly if you want (via the FFI) or you can write your own allocator using the low-level memory access facility.
Hiding in the low-level code in the FFI and in SBCL itself are "system area pointers:"
http://www.sbcl.org/manual/#index-int_002dsap
Finally, inline assembly. This is the least documented of the three features you mentioned, but here is an example of how to do it with the VOP facility:
http://pvk.ca/Blog/Lisp/hacking_SSE_intrinsics-part_1.html
Unfortunately, all of the above are extremely hard to use due to the lack of good tooling. Manual memory management is possible, but going from a pointer to a block of memory to a Lisp object is not at all straightforward (it is possible with a lot of low-level hacking using the sb-vm and sb-sys packages). You can define VOPs, but it involves a lot of steps; you are probably better off just using the FFI, but debugging across the FFI boundary is hard, and the FFI imposes a performance hit that might outweigh the benefit of your assembly language code.
Like I said, the problem here is tooling. I suspect that there is a lack of demand for these things among SBCL and CMUCL users, and in the Lisp community in general.
That's sort of besides the point, though, because the main problem with SBCL and other CL implementations is that they are written for use in userspace programs and depend on having access to kernel system calls and library functions like malloc(). So if you were to write a kernel in CL, you'd first have to write an implementation for CL that can run on bare metal. At that point, you might as well just implement a different Lisp dialect that only has the features you need in order to write the kernel.
It's just interesting that in theory a CPU could be designed that didn't have the concept of memory as a sequential array of bytes.
In fact, in the future this will probably rear its head again. Managing storage at a higher level could make things like massively parallel computing, caching schemes and virtual memory more powerful.
Perhaps one day the elaborate hoops current CPUs and OSs have to jump through to maintain performance in the face of memory latency limitations will be surpassed by something like this, and just as C is best suited as a systems programming language for a flat memory model, so a language like lisp will be best suited to low-level programming for this new architecture.
https://en.wikipedia.org/wiki/Lisp_machine
It would be interesting to see how to implement a Lisp machine for the modern day. For it to be useful, it should support some sorts of parallel-processing capabilities that conventional CPUs don't have. Perhaps pmap and preduce as SIMD instructions.
> For it to be useful, it should support some sorts of parallel-processing capabilities that conventional CPUs don't have. Perhaps pmap and preduce as SIMD instructions.
A Lisp Machine can be useful without that or with different architectures. In those days there were quite a few attempts to speed up Lisp. Thinking Machines sold the ultimate Lisp accelerator: the Connection Machine. Stuff like that was mind-blowing and mind-blowing expensive.
Remember, a useful Lisp Machine in the early 90s had a 40 Mhz cpu with 40bit architecture, a graphics accelerator for a second screen, 40 or more megabytes of memory, a 400 MB disk, 10Mbit/sec Ethernet, 5 Mbit/sec SCSI, ...
Today we get already multiples of those numbers for tiny money and a 64bit processor. Lisp runs nicely when implemented natively.
It's difficult to take the question seriously rather than as an attention-grab.
I think CL is in a way more operating system like than most other Lisps and would make good base for operating system. You can load and unload stuff, clean the system, etc.
The networking stack, device drivers, bootloader, etc, all need to be written in much the same way they would in a C-written kernel.