Systems Software Research is Irrelevant (2000) [pdf]
cs.bell-labs.com
cs.bell-labs.com
"Compatibility" has killed OS research, because the OS's main job is to multiplex the underlying hardware among applications, and it seems that the current OS's do that well enough.
I think that "systems research" has today moved towards building infrastructure on top of VMs (Java / .Net). There's a lot of innovation there which would have before went into a new shiny (and incompatible) OS. (Example: Java security policies.)
Also: writing a new OS requires a lot of upfront investment with _zero_ research value: boot/startup code, managing the CPU's idiosyncrasies, _drivers_ for bazillion of different HW units, a filesystem, TCP/IP stack, etc. No wonder people just tweak existing systems, as it's almost impossible to rip out these boring parts from Linux/*BSD kernels and reuse them in something new.
My personal wish for furthering system's research of today is something that would kill the need for hypervisors. Android, for example, runs each app under a different UID, thus achieving isolation w/o hypervisor. Let's devise something similar for desktops/servers; bonus points if the system can run existing binaries.
The only thing I'm aware of that does something akin to this is Rutkowska's Qubes OS.
You might like to look at Bromium (bromium.com), founded by a bunch of ex-Xen folks. I think they were doing something like this for the enterprise.
It's just a ton of system administration work. I've been playing with containers and chroots lately, and I actually can do exactly what you're asking for. Right now it's for servers (i.e. always run a mail server, DNS server, web app etc. in their own containers). But eventually I would want to do it for desktops, and sort out issues with X and sandboxing, etc.
My personal wish for furthering system's research of today is something that would kill the need for hypervisors. Android, for example, runs each app under a different UID, thus achieving isolation w/o hypervisor. Let's devise something similar for desktops/servers; bonus points if the system can run existing binaries.
This isn't really a research topic. It's mainly an implementation issue. It's possibly a standards issue, but I think giving each app its own POSIX environment with Linux namespaces gets you almost all the way there.
The main obstacle I'm having is that distros like to spew files all over file system. So although I could technically use raw .deb packages, I'm ending up making my own packages that are bettered suited to the "every application in a container" model.
So to make what you're asking for a reality, I think the things that need to happen are:
- Linux needs to get hardened more (i.e. right now it is not entirely safe to run untrusted apps unless you are adding something like seccomp, e.g. which ChromeOS contributed to the kernel for this use case)
- There needs to be the equivalent of Debian ("the universal operating system") for applications in containers. This just requires a lot of mucking with configure and build scripts. It's not hard, just tedious. Application authors have to learn to package a slightly different way.
Let's say I'd like to have a context menu where I can choose "Run in sandbox", then I would either choose a preexisting sandbox configuration or define a custom one. IMO, there's a plenty of research do be done on how users can seamlessly define allowed data-flows (what the sandboxed application can read and write), and how to implement the allowed data flows.
There's a fine line between "implementation issues" vs "research topic". Research can touch other fields than technical, e.g., HCI.
As an example, GPG is a powerful system, but all available UIs and "integrations" with existing mail clients are clunky at best. We don't know how to create a good crypto UI for GPG. There's a research topic.
Therefore, even the technically best sandbox won't be used if it entails a "ton of sysadm work". (For example, I have Linux installed in virtualbox VM, but I don't use it often because the integration with the hos OS -- e.g., clipboard sharing -- is clunky.)
For example, I'm working with a bunch of other people on MirageOS [1], which is clean-slate, library OS written entirely in OCaml and targets the Xen Hypervisor. Since Xen works on ARM devices, we'll also be able to create applications/appliances for the coming wave of embedded devices (aka Internet of Things) -- in addition to running things on the public cloud.
If you're interested in reading more, there's an ASPLOS paper [2] where you can find out more. There will also be a CACM article in the New Year that covers more of the wider background.
[1] http://openmirage.org and http://nymote.org/blog/2013/overview-of-mirage/
Yup, that's definitely a new area. I guess you're using the hypercall API instead of pretending to talk to the raw HW?
In a way, the hypervisor has become "the" OS, while "an" OS is now just an application running within the hypervisor. Can we avoid the additional (hypervisor) layer? I think yes. But: 1) how, and [harder] 2) also while maintaining compatibility with existing applications? (Not existing OS's, just existing applications.)
ps: I love the idea of operating system + ml. I remember BitC tried to bring types on the metal ~without success.
I think the talk shows how much can change in 10 years.
What would a talk like that today look like? What field would it be about?
And also, while he was writing this, what theoretical research was crystalizing that shaped today?
There is a problem with academia - it often is used by the government to answer industries inability to invest for the long term, rather than fixing the economic/legal system so that it is incentivised to. All this life sciences research oriented towards medicine going on at universities should be privately funded. Same with the other current booms in accademia like renewables. That they are not shows that something is broken in our version of capitalism.
The point about project size is also still valid. There is no way a small research group at a university can "compete" against Rackspace or Amazon in cloud computing innovation.
Seeing your reply to rictic, it seems you want something else? I'm not sure what that is. You want some automation to this sandboxing for each application? Either way, I think rictic was right to bring up the work that has been done in the Linux kernel with KVM, namespaces, etc. As one commenter on the LWN article points out, "with KVM we call hypervisors 'Linux'".
It's easy to miss that this was pre-LLVM. LLVM was an interesting development from the systems/design perspective more than the compiler technology one.