The benefits and costs of writing a POSIX kernel in a high-level language

Companies managing large houses with many thousands of computers care about efficiency gains in the sub-percent range - so even a 5% performance penalty is entirely a no-go. I doubt the kernel is nearly as featureful as Linux, either.

Now, could one of these academic kernels have sheer engineering effort put into optimizing that 5-15% regression away? Probably.

Surely that's a practical position to take, but as an idealist, thinking over the long term... wouldn't you rather have that extra performance in addition to a lack of security vulnerabilities? There's no fundamental trade-off between this kind of performance and security.

I don't think a lot of people will see this the same way you do. I think a lot of people will care about even a 1% degradation in performance. And people will prefer a tried and true operating system for security over something that hasn't been tested in the real world, even if it's designed to eliminate one class of security vulnerabilities.

For what it's worth:

The conventional wisdom shared by many of today's software engineers calls for ignoring efficiency in the small; but I believe this is simply an overreaction to the abuses they see being practiced by penny-wise-and-pound-foolish programmers, who can't debug or maintain their "optimized" programs. In established engineering disciplines a 12% improvement, easily obtained, is never considered marginal; and I believe the same viewpoint should prevail in software engineering.

Donald Knuth. Structured Programming with go to Statements. (It's the same paper that the oft-misquoted "premature optimization" quote comes from.) http://www.cs.sjsu.edu/~mak/CS185C/KnuthStructuredProgrammin...

>(For the same reason, I want the standards committee to produce a bounds-checked dialect of C where I can choose to pay some perf cost to get real dynamic and bounds-checked arrays)

The Checked C project is working on such a bounds-checked dialect of C: https://www.microsoft.com/en-us/research/project/checked-c/

> I know I'd trade 10% perf for the elimination of whole classes of security vulnerabilities.

Firstly, I would like to know if that's an average of 5-15% slowdown, and if that means some operations are multiple slower and some are close to equivalent. Average means nothing if your workload triggers the pathological cases.

Secondly, you might be giving up classes of security vulnerabilities and gaining classes of kernel DOS bugs. I don't write kernels, but my understanding (mostly from comments of people here that do), is that kernels rarely allocate or de-allocate after book, as they immediately reserve the memory needed for their limits and then work within those limits.

Believe me, I'm not a proponent of C. I'm critical whenever it comes up, as I think it's caused a huge amount of harm. That said, of the places where C has a well deserved reputation of being the right tool, I think kernel development is one of them. That isn't to say other languages don't have something to offer, but this is one area where you actually seem to need all the crazy unfettered access that C gives you.

> An unoptimized research kernel

The authors would probably not call it "unoptimized". From the last paragraph in Section 8.8:

"It took effort to make Biscuit achieve this level of performance. Most of the work was in understanding why Linux was more efficient than Biscuit, and then implementing similar optimizations in Biscuit. These optimizations had little to do with the choice of language, but were for the most part standard kernel optimizations (e.g., avoiding lock contention, avoiding TLB flushes, using better data structures, adding caches)."

I agree that it's impressive, and better than I would have guessed.

It's on a quad core machine with a tiny heap to manage, even OSes like OpenBSD do alright with Giant kernel locking at that level of concurrency.

This would turn into a massive research project on GC and language runtimes where the commodity server market already is and especially where it is going considering a dual socket AMD Rome will have 256 threads and a single socket POWER10 would have the same. To consider that project in terms of time scale and cost, one can look at the history of JVMs. Go's type system might make some things slightly easier but there is no implicit scalability codex they've unlocked beyond state of the art JVMs that have been evolving for 15 or more years.

This is why Rust brings something novel to the table for systems programming. With the borrow checker, memory safety can be added in many cases without runtime overhead and follow on contention.

An extended though, we need to be moving _toward_ sympathy with the on chip network and implicit message passing nature of cache coherency policy. This is mostly a library/ecosystem shift rather than language.

It would be more impressive if this wasn't an uphill battle since the late 70's against beliefs.

We would be further down this path if bigger companies really cared to push down this path no matter what, but politics and short term profits always win in the end.

> I want the standards committee to produce a bounds-checked dialect of C where I can choose to pay some perf cost to get real dynamic and bounds-checked arrays

If you don't want to wait on the standards committee, the SaferCPlusPlus library provides memory-safe substitutes for commonly used unsafe C/C++ elements (including pointers). These memory-safe elements require a C++ compiler, but because they are largely compatible with their native counterparts, you can write programs that can switch between using memory-safe (C++) elements and (unsafe) straight C elements with a compile-time directive.

That is, you don't have to choose between safety and performance when authoring the code. You can make the choice at build-time. So you could, for example, provide both "memory-safe" and "non-memory-safe" builds of your software. There was even a (now long neglected) tool[1] to automatically replace arrays/buffers in C source code with (optionally) memory-safe substitutes. (Not just bounds-checked, but safe from use-after-free.)

> I know I'd trade 10% perf for the elimination of whole classes of security vulnerabilities.

Full memory safety may sometimes cost a bit more than that [2]. But a nice thing about this approach is that you can choose to use (faster) unsafe code in critical inner loops, and memory-safe code everywhere else.

[1] Shameless plug: https://github.com/duneroadrunner/SaferCPlusPlus-AutoTransla...

[2] https://github.com/duneroadrunner/SaferCPlusPlus-BenchmarksG...

Production ready means different things to diff people, and I don't know how many people use it in Prod, but have you looked at redox:

https://github.com/redox-os/redox

I would expect that slowdown to be more a consequence of optimizer implementations than anything else, given the quality of C compilers code about 30 years ago.

Is the code on GitHub?

For the ”C kernel vs go kernel” benchmarks, the paper writes ”The benchmarks allocate no heap memory in steady-state, so Biscuit’s garbage collector is not invoked.”.

Is that a fair way to make this comparison?

Also, ”This results in heap bounds of less than 500kB for all system calls but rename and fork (1MB and 641kB, respectively).”

How does that compare to the C-based kernel?

Finally, how does this system hold up after days of uptime?

I'm not at OSDI or I would have asked there, but:

1. Your skepticism for using Rust cites [26], the PLOS paper from the same people who did [27]. They've essentially recanted, in that their PLOS talk was right after Niko's keynote, and he then talked them through how interior mutability works in Rust. Other than [26], are there other lingering concerns? I could imagine so, but it would be great to take stock.

2. You don't seem to page to disk, and do OOM shootdowns instead. If you did enable disk paging, how comically bad an idea does a tracing GC become? :D Naively, I could imagine RC-GC being less horrible, but perhaps that's my built-in (and absolutely for real) bias.

Non sequitur, but is Cody Cutler (you?) related to Dave Cutler, the designer of VMS and Windows NT? Maybe OS programming runs in the family :)

"Production" - they won't commit to actually supporting it for production for the rest of us (currently got app engine flex burning a hole in our pocket at work because the old runtime was too problematic and sans it being out of beta we can't move to the new one).

It is missing a lot of stuff that is potentially time consuming, like access control.

GC enabled system programming languages like D, Active Oberon or Modula-3, among others, allow for many ways to allocate memory, it is not GC all the way down.

Thats the sad state of (i guess) automatic pdf text extraction, consequence of research papers being exclusively in pdf, consequence of (la)tex coming from another age. I love and praise tex for what it allows to do, but my opinion is that now is the time to get past it, learn everything it has done right and apply the new knowledge we have in language design to get a better surface language (lower friction syntax, higher-level semantics allowing to separate structured content from typography and extract other stuff than a visual document from a source file). Tex being so good means it has such a monopoly that this kind of project have to be tremendously good to have a chance (which is probably a good thing).

Last time I looked (which was a while back) both Go and Rust binaries for "Hello world" were over a megabyte with out-of-the-box settings, compared to just a few kilobytes for C and slightly more for C++. I should qualify that they were the simplest of Hello world programs, I didn't add any fancy options to try and reduce the size at all.

I'd be keen to see what the smallest binaries the reference implementations of these languages can produce are.

The same that happens to the Linux kernel I guess.

This is covered in the paper. They use static analysis to know the memory costbof a kernel function and rather than optimistically run the kernel function and then unwind it if they run out of memory, they delay it until memory is available.

I think it's a moving target. 70 years ago, assembly was high-level.

Though implementing fork() in a kernel is entirely distinct from using fork() in user space. Implementing fork() is mostly "just" manipulating a bunch of data structures the right way. Copy the right bytes and set the right values.

It's like how you can implement Java, C++, python, Haskell and any other (practical) language in C, despite C having no support for e.g. classes or lambdas. Or how humans can design and build combustion engines, despite us not having the capability to ingest gasoline in order to rotate an axle at 6000rpm ourselves.