Hacker News Reader

There is a constant dance at the fringe of high performance systems. It leads to a recurring pattern of "revolutionizing" with some kind of bypass or coprocessor architecture, then eventually reverting to traditional structures as the new performance realities reach commodity levels.

Part of it is the economics at the fringe can pursue speed at any cost. And part is the heady appeal of doing things differently for researchers and advanced practitioners. But, in the long view, I think you are right that it is a bad idea. If you care about maintenance and sustainability, you usually find that these bypass solutions get abandoned as soon as the more conventional approaches can approximate their speed on newer commodity hardware. So there is huge churn in these specialist devices with specialist APIs and tooling.

There is a recurring theme in high performance networking where crazy things are tried and all sorts of fancy protocol offloading written, then eventually deprecated because it is seen as a support burden and a source of bugs. Because each of these specialized stacks has a smaller user base, they are have less economy of scale to invest in maintenance and stabilization.

> There is a constant dance at the fringe of high performance systems. It leads to a recurring pattern of "revolutionizing" with some kind of bypass or coprocessor architecture, then eventually reverting to traditional structures as the new performance realities reach commodity levels.

Truly this is what DC hardware been about for 20+ years.

Look at HPC space, and than look for "uncle Joe hosting." The moment Joe starts looking for HPC and invest heavily for some super duper RDMA powered disk, a mainstream hardware maker comes and spoils everything by releasing a mass market product doing just that if not better, thus ruining the grand scheme of gaining competitive advantage through some "secret sauce hardware"

Look at video transcoding or that new "AI" neural network thing. After few years of entertaining themselves with idea of purpose made hardware, big boys simply went the route of just using a lot of mainstream hardware.

I don’t think anyone is suggesting that every application explicitly code for each network device. All of that tricky logic could be put into a userland library instead of the kernel. If we wanted, we could even replicate a kernel device driver style API in userland that network device drivers program against, done as a set of userland library files loaded dynamically based on detected hardware.

The tricky part wouldn’t be sharing code between applications. We know how to do that. The hard part would be figuring out a clean way to share the hardware between all running applications, given that any app could be terminated at any time, apps might be mutually untrustworthy and apps would have to play nice to share resources. I can imagine a hybrid approach where the kernel allocates network queues to applications and suggests userland device drivers. While running, the apps would have direct access to the hardware. And when the app is terminated the kernel would reclaim the assigned hardware for reuse by other applications.

Microkernels tend to do a lot of what you're talking about. They accomplish this via interrupts and virtual mapping of the device memory to userspace for the userland app to access.

"If we wanted, we could even replicate a kernel device driver style API in userland that network device drivers program against, done as a set of userland library files loaded dynamically based on detected hardware."

This is how many microkernel based embedded RTOS's work. Common API's written for a hardware device, say a NOR flash chip, and the driver for the hardware onboard implements the interface specific to that hardware. You can then dynamically link the driver specific to your hardware. That's what I was doing as an embedded C developer some 10 years ago, and that solution had already been implemented for at least a decade (or 2) before that.

You should take a look at GPUs.

All of the complexity of 3D rendering is implemented in userspace, and that has been the case universally in production for more than 5 years now -- closer to 10, really. If you replace "all" by "almost all", you can go back much further than that, really all the way to the beginning of GPUs' existence. And yet normal application developers don't have to care, because the driver does it for them.

The point is, drivers don't have to live in kernel space, they can live in user space as well. Networking folks may start being more serious about this nowadays, but it has been the reality in GPUs for a long time.

Disclaimer: IANA kernel developer.

Is there any benefit to having those specialized developers create frameworks or libraries in Userland which other developers can leverage? This way they remain the interface to uncooperative hardware, but the code is in Userland so the bold folks can try their own approach.

Seastar.io is another one, it's an async engine that utilizes all of the cores in a modern system and is the heart of Scylla's DB. However, it's complex to use it correctly

That's basically DPDK / SDPK.

At least for NVMe SSDs that is already solved, there is a standard that all NVMe SSDs implement and you only need one driver for all of them. If you want you can use SPDK or one of the few other drivers and you get the full speed block access.

What you don't get however is sharing the disk between multiple processes.

As I was reading this, I remembered the days of my youth setting the IRQ and DMA address for my soundblaster (compatible) soundcard.

Be sure to avoid IRQ7. That conflict with LPT1 and/or LTP2 can be brutal.

Haha, me too.

> Hardware vendors don't like to share. They don't share code, they don't share common interfaces, they don't even share documentation. As it is now, these are all problems which most userland developers don't have to care about -- those problems get dealt with in the kernel, by developers who specialize in building support for uncooperative hardware.

Thus more money for us :) I think this fact almost begs to be taken advantage of. See, how things work in corporate storage products, with EMC being the prime example

I enjoyed the paper. My impression is that you’d shift the burden to the runtimes that, for many applications, currently sit between POSIX and applications (e.g. see the Q&A about POSIX).

Considering the poor quality of userland software, this seems like a terrible idea.

Anyone willing to spend the money can have top quality userland software.

Getting a better OS is not always accessible, even with deep pockets.

An OS is more than just a kernel.

OS’s would simply ship with user land drivers instead of kernel space drivers