Lisp-based OSes
linuxfinances.info
linuxfinances.info
Beyond that: the OP sez that "if the whole system is constructed and coded in Lisp, the system is as reliable as the Lisp environment. Typically this is quite safe, as once you get to the standards-compliant layers, they are quite reliable, and don't offer direct pointer access that would allow the system to self-destruct."
But as I write, we've got two Lisp posts on the front page, and the other one[1] is about the performance of code compiled with
(declaim (optimize (speed 3) (safety 0) (space 0)))
So, at the very least, the "quite safe" guarantee applies only to code compiled with full safety checks, which typically come with a very large performance hit.
I don't know if he's right, but that seems to be his point.
Determining which hardware path to use to add two numbers would be done in the hardware itself. Check the type bits of the numbers and feed it into my ALU. Compare this to an x86 lisp, or compiled C, where 'type' of a 'number' is determined by the assembly code instruction that is used on it.
This isn't just a performance improvement, it is also an improvement in the safety of the dynamic language.
There are a lot of different things that you could do for garbage collection. You could have in-hardware reference counting, or 'dirty' and 'clean' (or color) bits, for a mark and sweep collector, or 'generational' bits for an ephemeral garbage collector.
The idea is that any time you take something out of software, and put it into specialized hardware, you should get a performance improvement.
This doesn't mean that the lisp on a chip would be faster than C on a comparable x86 chip, it means that the things that make lisp (and other functional languages) safer and easier to use would be supported in hardware-- therefore not slowing things down as noticeably.
Or, at least, forcing every language implementation on that hardware to use the same safety mechanisms, making some apples-to-apples benchmarks impossible.
It would be interesting to see what a C implementation for that hypothetical modern Lisp machine (CADDR?) would look like.
A close parallel is AMPC, which compiles C to JVM bytecode.[1] The vendors say it's standards-compliant, and I actually am pretty sure it is, but it doesn't do a lot of the nonstandard things C programmers have come to depend on. For example, the 'struct hack', where you pack data of multiple types into a struct and proceed to index into it as if it were an array (usually an unsigned char array), flatly does not work, due entirely to the runtime type checking done by the JVM. This always seems to lead to major debates over whether it's a very good compiler.
What causes problems is casting pointers to ints and back, and casting all other crap to chars. This is not standards compliant.
Casting ints to pointers will never be type-safe, but one way to get around that is to just ignore the cast, and overload arithmetic operators to work correctly on pointers - the pointers will carry around their type info, and everything should work ok.
Casting other crap to chars will never work because it interferes with the way the other crap has its type encoded. Luckily in most cases this casting is done to perform I/O, where you can also just ignore the cast, and specialize the lowest-level I/O functions to dispatch on the actual types.
The moral of the story is that you should basically ignore all the line noise the programmer produces about types, and look at the actual objects. This is exactly how Java works, btw.
WRT hardware tagging and type checks, there's really no reason to do it on a byte-addressed superscalar processor. If you look at 64-bit Common Lisp implementations today, you'll actually find that they use only about half the available tag bits in each word. The only thing that needs to be boxed is double-floats.
Unix processes virtualize real processes by providing indirection to the basic state identifiers of machine code - memory addresses. It's actually much simpler to virtualize a Lisp image - the basic identifiers are symbols. Dynamic binding already lets you do this to parts of the Lisp process. What's needed is an extension to let you do this for packages/modules.
"That is --- "omit safety checks, just trust me that my array accesses are all in bounds and I'm getting the types right." Code compiled this way is not inherently safer than C, and has to be coded up with equal care.
So, at the very least, the "quite safe" guarantee applies only to code compiled with full safety checks, which typically come with a very large performance hit."
With the Common Lisp model of optimization you can do this selectively for parts of your code, and the compiler can ignore your type hints or declarations if it wants to.
Array bounds checking can be lifted out of loops because CL arrays come with information about their dimensions. This is a fundamental and unfixable problem in C.
Type safety checks are only a performance problem in numeric loops, where it's always possible to provide the compiler with type information so that it can produce provably correct code (if you're looping over arrays of floats, you only have to check the type of the array once outside of the loop - again, this is a fundamental problem of C that is unfixable).
People have wasted hundreds of man-years trying to come up with automatic verifiers and safe subsets of C, but the language has so many problems and so few upsides that none of that work is worth it if you want to build reliable systems. Everything about C that is broken is actually very easy to fix, as long as you don't mind not using C.
What if those parts of your code have bugs in them? There's no reason they wouldn't, and whoever's writing file format parsers (fonts, video, JPEG) on your platform is going to turn off safety because it's too slow, even though that code has many attack surfaces.
1. Those type checks will be too slow
2. The compiler will choose to follow declarations for untrusted code
3. The declarations will lead to bugs on some inputs
4. Those bugs will be exploitable
5. Somehow those exploits will be worse than what's currently the case with C
I honestly don't understand the people that argue bounds checking is too slow. How many times do you have to make the same mistake to realize that what you're doing is wrong? It's just idiotic.
Around campus, it's been described as an opportunity for Shivers et al. to write a Operating System built completely with functional languages, from the low-level drivers up to user space tools and new programming languages.
My personal thoughts is that it'd be awfully cool to have something like "Emacs as a real OS." Perhaps it is lack of knowledge and self-confidence or the limited nature of Emacs, but I find it way easier to change the way Emacs works than to change the way the Linux kernel, GNOME, GNU tools, etc. work.
[1] Page 12 of http://www.ccs.neu.edu/news/CCIS-Newsletter-Fall-10.pdf
- Replace elisp with Common Lisp
- Build os-level threading support
- Build a hardware abstraction layer / target a 'bare' machine.
That gets someone a 'ways' towards a traditional Lisp OS.
I think one of the big questions that arises in for a modern Lisp system is the design of of multiple processes and multiple users.
Those last two would certainly be important as well, but something I have no context on.
That all said, I prefer the modern world of general purpose operating systems with good commercial (Franz, LispWorks, etc.) and free (SBCL, Clozure, Clojure, etc.) Lisp development environments.
Smalltalk implementations are pretty similar in this respect, although both language families are still pretty fun. Erlang and Inferno/Limbo are both sort of guilty of this, but for slightly different reasons. But it's pretty inconvenient around the borders, getting into and ot of the "real" OS. It's sort of a mixed blessing that nowadays, most developer OSs are (on some level) Unix: there's a baseline, a set of facilities that can be expected to be present and to behave a certain way, so most modern Lisp-like languages are pretty friendly to the OS (or at least provide an FFI to C). The downside is that the baseline is POSIX, a little stiff sometimes and often quirky for historical reasons. (Still waiting for Plan 9 to catch on...Maybe I should stop holding my breath.)
A problem with such environments might be the availability of widely used applications like Google Chrome or Firefox. A way around this might be to expose the language's Virtual Machine as bytecode or some other intermediate, and target C compilers to that. This way, an entire POSIX environment could be built on top of the Lisp based OS, which would be more comfortable for many users, yet still offer an omnipotent, seamless access to code everything "from the bare metal-up" in Lisp.
For another thing, no one would be building server/back-end infrastructure on top of such a POSIX layer. It would be there for compatibility for end-users. (So everyone could have Firefox, if they happen to need it, which isn't going to be that common anyhow. Most such machines would be headless.) If you want to run things for heavy-duty functionality on POSIX, just put it on another machine in the cluster running Linux. The whole point of such a system would be to code servers and such functionality in Lisp with exposure of the bare-metal as 1st class Lisp entities.
Performance won't be great but for the purpose it would serve, it certainly wouldn't be terrible.
A few weeks ago, Azul's VM was on HN, and its GC benefits from tight integration with the virtual memory system.
http://www.managedruntime.org/
From what I understand, the main win is that they use nested page tables to let the JVMs handle page faults directly, which is how they implement high-performance read barriers.
I don't know a lot about garbage collection, but read barriers seem to be the essential piece for implementing real-time (which really should be called "non-blocking") GC.
There's a good discussion on LtU about this: http://lambda-the-ultimate.org/node/4165
[edit] I should mention how this relates to Lisp operating systems: if you replace the virtual memory system with a garbage collector (ie push the GC into the kernel), you can get the same effect but without needing nested page tables/VT-x/RVI, even for user-space processes.
It should also be more efficient and waste less memory on fragmentation than going through a dumb VM.
Although I don't know much about midori
Frode V. Fjeld doesn't seem to have much time to hack on it anymore, but the mailing list is active and you can hack on it today.
Since people aren't willing to do this it goes to show that the claims of the Lisp junkies are just pipe dreams.
If programming on an all Lisp environment really was 10x more productive even a ten thousand dollar price tag would be chicken feed.
Lisp fans like to talk it up about how great it is, but at the end of the day are unwilling to put their money where their mouths are.