Hacker News Reader

Comment by npsimons | Hacker News Reader

Been done before (some would say to death), and the reason we have memory protection between applications is forgotten because people don't realize how nice it is. Sure, sure, your big "well engineered" web browser needs direct access to the hardware for speed, but painful experience has taught us that giving apps programmers direct access to hardware is a recipe for failure. Besides, there are already plenty of workarounds to get faster (eg, mmap) or even direct access to hardware from userland, not to mention the myriad of virtualization and protection schemes and levels in userland (eg, SELinux). This seems like a solution in search of already solved problems. Although as a research project, it does seem interesting . . .

View on HN

Arrakis doesn't eliminate memory protection or process isolation, nor does it give applications direct hardware access. It takes advantage of modern hardware virtualization capabilities to present certain virtual devices to the guest applications, eliminating the kernel from the I/O path for critical operations like receiving a packet.

Read this paper:

http://arrakis.cs.washington.edu/wp-content/uploads/2013/04/...

Good points. Indulge me for a minute while I try to formulate what they might say in response: most of the complexity of an OS is device drivers.

You're right, the basics of process isolation, scheduling, and resource allocation are pretty dull.

Maybe the UW project makes the move to userspace official? At 1,000,000 packets per second it is more efficient to put the whole network stack in userspace. Assume similar gains can be had for all types of hardware, and shared libraries eliminate the overhead of loading the whole network stack every time you run 'ping'.

That's just a 30-second pitch for the idea. Userspace is not new (see: Mach). I think what is changing is the average lifetime of each process. Both desktops and servers are beginning to become a commodity to the point where the old reasons for not using a microkernel are starting to become irrelevant: e.g. message passing cost isn't as much of a problem if the file server runs 'smbd' 99% of the time and that is the only really performance-sensitive application. Likewise for a desktop, except the running process is 'Chrome', or for a tablet running 'dalvikvm'.

Really it's a move away from code complexity that wasn't possible before but is becoming possible.

So are we talking Microkernel here or something else? Sorry to constrain by categorizing, but it's what I do sometimes :) and I am thinking of some OSes (mostly research/academic) that went this way (of no layers between userland and the hardware) that weren't Microkernels, but I can't remember any names right now. I have to admit, part of what keeps my interest in this is that I used to dabble in OSes and I haven't gotten to for quite some time :(

EDIT: Exokernel was what I was remembering (I think; https://en.wikipedia.org/wiki/Exokernel). I can see some of the advantages, but as a current maintenance programmer and previous RTLinux hacker, I have to say I have my doubts about giving most programmers direct access to hardware :) That being said, giving programmers (and admins!) easier access to containerization, virtualization, or just intra-process protection in general would be a good idea. As I've said, there are plenty of options right now, but none seem to be well documented or advertised. There was a CACM article a while back that predicted a future in which you had one smartphone, but it had split domains for work and personal apps and data. Interesting stuff, and as you say, finally becoming possible.

Now that more knowledgeable people have chimed in, it looks like Arrakis is an exokernel that only needs hardware virtualization to isolate domains.

The idea behind exokernels isn't that you give "most programmers" direct access to hardware, it's that you give a library writer direct access to hardware, and then programs link in the libraries that are most appropriate for their application. Basically, separate protection from policy & abstraction: the kernel still provides protection, but the library handles all policy & abstraction details. So instead of having your database mimic disk blocks on top of a filesystem, just expose the raw disk blocks to the DB. Instead of having your garbage collector fight the virtual memory system, expose the TLB and physical pages to the language runtime. Instead of requiring that every read() go through the kernel, have the NIC write directly into application buffers.

The big change that makes this desirable now but undesirable before is that machines are becoming increasingly single-tasked. When everybody owned a desktop and had a dozen programs on it, you needed to write an abstraction that could support all of those programs. When everybody just runs a web browser that connects up to a web server in the cloud which connects up to appservers and databases running on separate machines, each machine only needs to support one type of application. So let that app link in the abstractions it needs and only those abstractions, ensure that existing daemons don't trample over each other, and have the kernel get out of the way.

> So instead of having your database mimic disk blocks on top of a filesystem, just expose the raw disk blocks to the DB. Instead of having your garbage collector fight the virtual memory system, expose the TLB and physical pages to the language runtime. Instead of requiring that every read() go through the kernel, have the NIC write directly into application buffers.

This sounds great. Do you know of any literature/writings on these kind of things for programming languages, specifically? For example the thing you mentioned about language runtimes.

There's little about exokernels because to my knowledge no exokernel has had widespread production use. Lisp Machines had a garbage collector that was integrated into the kernel's pagefault system, though, which meant that most collections would not trigger page faults:

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.125...

> Been done before (some would say to death)

How is that George Santayana thing? "Those who cannot remember the past are condemned to repeat it"?

I like seeing new ideas being tested in the OS arena. It's a real shame the two dominant OSs these days are Unix and VMS. I refuse to believe these two are the best humans can come up with.

From "The Rise of 'Worse is Better'", written in 1989:

"The good news is that in 1995 we will have a good operating system and programming language; the bad news is that they will be Unix and C++."

The predicted date was a little early, but otherwise he pretty much got it right. I think that, from a late 80s Lisp perspective, modern Windows fits into the "UNIX" category, and Java/C#/ObjC/whatever are close enough to C++ to count.

>The predicted date was a little early

wouldn't say so - my first Linux machine (slackware, shoebox of diskettes :) i installed exactly in the 95 :)

>from a late 80s Lisp perspective, modern Windows fits into the "UNIX" category

while psychotropic mind-altering effects of Lisp are well known, i'd doubt that it can produce such perception changes or even in such a direction.

Windows and UNIX are not so different. Monolithic kernels written in C or C++, permissions largely done with user granularity, byte-addressed memory, virtual memory with per-process address spaces, no hardware support for tagged pointers or garbage collection.... Compared to the variety that's gone before, they look nearly identical in their fundamentals.

Thucydides?

I would go even further and say that apps developers don't want to write a networking stack, display driver, roll their own sound implementation, etc.

It sounds like the "bad old days" of DOS where games had to have their own implementation of these things.

That's why Arrakis provides that functionality through a library.

Reply on news.ycombinator.com