The Demikernel Datapath OS Architecture | Hacker News Reader

The Demikernel Datapath OS Architecture

micahlerner.com

38 pointszdw9 comments

hey, sujay (one of the authors from the paper) here! happy to answer any questions about the paper/system.

Well, not a question.

POSIX and Linux are not the same. There is no epoll in POSIX, it's a Linux addition. The BSDs have kqueue which according to some people is actually better [cf. https://www.youtube.com/embed/l6XQUciI-Sc?start=3362 ]. But kqueue isn't POSIX either. Only poll() and select() are...

Yet another academic work in the OS space that should have been tested against the the state of the art in (commercially backed) networking space.

From the abstract:

> However, none of these systems offer a general-purpose datapath OS replacement that meet the needs of µs-scale systems

One of the key points that the authors use to motivate their work is that using kernel bypass technologies is hard because it requires software engineers to rearchitect their code.

Call me sceptical of that claim. I can think of 3 (open source) systems that provide drop-in replacement network interfaces for existing software, but offer nanosecond (not microsecond!) scale performance.

- Solarflare/Xilinx Open Onload - https://github.com/Xilinx-CNS/onload

- Exablaze/Cisco - ExaSOCK - https://github.com/cisco/exanic-software

- Melanox/NVidia VMA - https://github.com/Mellanox/libvma

With each of the above systems, they are designed to offer low latency network stack replacement for existing applications, without code modification or even recompilation. Exactly the authors’ goals.

But there is not a single reference to any of these technologies in the paper. And crucially no comparison to any of them in the evaluation.

The authors seem blissfully unaware of what work has actually been done to solve these problems. Which makes the contributions of the paper … somewhat dubious.

One crucial difference I was able to spot is that Demikernel seems to have an approach that can work for OSes other than Linux. To wit, examples of the performance of the Demikernel approach are shown on a Windows system running in Azure.

Of the three examples you listed, none of them seem to support their kernel bypass capabilities in anything other than Linux.

Fair enough. My guess is that the commercial solutions are simply going where the money is. If there was a commercially pressing need to port to other systems, I’m sure it can be done.

With that said, it's one thing to have a legitimate reason for preferring an alternative approach. But the paper doesn't even mention these solutions, which means that the authors are either ignorant of what's already been done (bad), or deliberately avoiding comparisons to alternative approaches (worse).

I would put the reviewers on the spot actually, even if the authors weren't aware of such solutions, the reviewers should.

This is the trouble with submitting to OS conferences. Their networking knowledge is typically limited. I’ve seen the same thing happen in the past.

Note that Solarflare requires a per NIC license to get that by-pass. And like Melanox those NICs are not cheap. Kernel by-pass on COTS without recourse to high end NICs w/ or w/o FPGA and the like is needed. I am benchmarking eRPC built on top of DPDK on AWS to that end [1]. And not having to play sys-admin and figure out configs or at least simplifying that work would help. DPDK isn't a necessarily straightforward library. [1] https://www.usenix.org/system/files/nsdi19-kalia.pdf

FWIW - I don't think it would be that hard to port VMA or ExaSOCK to run on top of DPDK, certainly less effort than making a whole new category of operating system!

But ... selling the hardware is where those companies make their money, so there's not much of a business case for making your free and open-source software run on other people's (cheap) NICs.

Reply on news.ycombinator.com