Linux: IO without entering the kernel

SPDK follows the same pattern as DPDK (for networking) but it has much less of a practical use case in my experience. You can easily outperform mmap() etc using io_submit() interface that already exists, saturating the capabilities of fast storage hardware, and the overhead of io_submit() is nominal if you are using it well. SPDK won't meaningfully increase your I/O throughput and the API is significantly more difficult to use. I've played around with SPDK in database storage engines and I find it to be inferior for the purpose compared to io_submit() in practical implementations. The number of IOPS required to make SPDK worth considering would be indicative of a more fundamental design flaw in your storage architecture, at least for the foreseeable future.

DPDK, by contrast, can dramatically improve networking throughput and performance. While also not the most friendly API to use, it undeniably improves CPU efficiency for packet processing and without an obviously better alternative.

This is what DSSD did since ~ 2014; drive NVMe directly to the D5 from user app, never incurring a context switch (in fact the D5 was mapping into memory on all the PCIe connected clients). Latency was astonishingly low, dominated by PCIe switches for writes and NAND reads for, eh, reads. EDIT: typo

If you are coding your application against libaio chances are the entire point of the system is to run that application.

High performance software generally does divide the system up so that certain cores serve certain roles. i.e IRQ handling etc. This is often done to exploit NUMA architecture, ensuring that memory accesses only happen against the local memory controllers and locally attached PCIE where possible.

If you are going to this point you are rarely if ever concerned with sharing the system with anyone else.

I don't know if it is the case, but, I would happily trade off two cores to get better IO in most cases in our nodes servicing IO bound jobs. Otherwise, those CPU cores are just idling anyway because of IO..

With the current growth of CPU leaning towards more and more cores with ever smaller improvements in single core performance, dedicating spare cores for such perm improvements is not at all a bad idea.

It's not at all unusual for our HFT and other realtime customers to dedicate cores to particular tasks. They often boot with isolcpus to reduce the number of CPUs visible to the scheduler, then dedicate those CPUs to servicing interrupts or running critical programs. They also turn off all power management so the CPUs are burning at 100% all the time.

This is basically how the entirety of DPDK using a poll-mode driver works. Each worker thread is doing busy polling on a shared memory segment, ensuring minimal packet processing latency.

If the AMD leaks are to be believed, we will have cheap (less than about $400) 16 and 12 core consumer processors next year. I’d happily trade two of those cores off against syscall overhead.

Dedicating cores to specific server tasks is normal in modern high-performance software architectures. It can very substantially improve throughput of the system. For server code generally, and data intensive software specifically, there is a reasonable and valuable presumption that your hardware resources are dedicated to your task. This assumption greatly simplifies the design of high-performance software and tends to match the deployment environment of performance-sensitive software in any case.

Many high-performance software applications are constrained by network/disk bandwidth, so you would have cores doing no productive work regardless. This allows you to re-deploy those unused cores in other creative and useful ways. Specialization of cores has the added benefit of minimizing lock contention and making it simpler to reason about complex interactions between threads.

Yes. But actually you don’t only donate the cores to the application but also the IO device which is acted upon (E.g. the drive or network adapter). This also makeshift sense for some very specific high performance tasks, but not for a general purpose system.

There's a concept called exception-less syscalls, which might be similar to what your thinking.

There is mmap for read/write (some of the most used syscalls).

There is also the vdso page [1] for things like gettimeofday etc.

[1] http://man7.org/linux/man-pages/man7/vdso.7.html

That's almost literally IOCPs.

I had this exact idea long ago, the syscall and interacting language would have to be different due to a mismatch, a graph language of some sort to describe dependent calls and batch and a language that makes async less painful. The non obvious benefit is a compiler could optimize across protection domains.

Talking with GPUs works a lot like this using modern APIs. You fill a command buffer with stuff and then asynchronously get results back "later", using synchronization primitives and memory mapping. There are syscalls to kick off the command buffers but those can (and often are) be implemented in userspace by the driver.

I have seen this technique being used to implement syscalls for processes running on intel SGX [0]. In that cases, it makes even more sense because exiting SGX, making a syscall and then reentering SGX is a lot of overhead for a single syscall, while writing to shared memory is doable without exiting the SGX environment.

[0]: https://www.usenix.org/system/files/conference/osdi16/osdi16...

I believe that describes the L4 messages. So, yes, it can be generalized, and there is a widely used kernel family out there that only uses it.

But you will lose some performance on some of the syscalls. While an interruption is expensive, memory access is expensive too. When you have only one of the options, you will have stuff you can't optimize. The thing with IO is that the slower part of the memory access can be done by a co-processor, so you get the entire CPU available for more important work.

Performance, performance and performance.

Someone more familiar with kernel workings than me should clarify, but my understanding is that IO generally happens via a syscall which requires the thread/process in question to context switch between userspace and kernel space, which can be very expensive. By enabling IO polling in userspace, you get to avoid that context switching.

The motivating benefit is performance, but a side one the author mentioned on Twitter https://twitter.com/axboe/status/1073320502532263936 is sidestepping Meltdown and similar vulnerabilities from having the kernel and the OS in the same address space (even though they're separated by a privilege boundary). In a scheme like this, you can theoretically dedicate one core to the application and a separate one to the kernel, and minimize speculation, cache sharing, etc. between the two. The application and the kernel share a portion of memory, so the kernel doesn't ever run on the application's CPU.

This is questionably practical for a general-purpose machine, but for a server system used entirely as a hypervisor, or web server, or file aerver, or something, it might fit really well.

In traditional I/O, a hardware interrupt is triggered whenever data arrives at hardware boundary and the interrupt can get serviced by any core that is available to the scheduler. One can imagine how much overhead is involved in context switching whatever that core was doing before, setting up the registers, moving data and then relinquishing the core back to OS - in this model, dedicated cores serve I/O in a memory mapped ring buffer like data structure sized to your application needs. There is no allocation/deallocation overhead, no management beyond moving a pointer and no context switching. If you can spare the cores, this can significantly improve performance.

In one use-case, I was able to quadruple the performance on a 32 core xeon by installing 4 10gbps ethernet cards and dedicating the first eight cores to I/O (2 per interface). This is all about latency but with proper care, it also improves throughput.

I've never totally followed why AIO is its own set of interfaces instead of a generic async system call mechanism based on something like this. Write a bunch of requests to a page in the form of the registers that would make up the syscal, then wait for one to be completed using a futex or something (where futex(2) remains an actual system call).

Morphed into Midori, which never made it past being a research project. http://joeduffyblog.com/2015/11/03/blogging-about-midori/ has some posts about other things they explored in Midori.

JX OS took that approach for most of its protection with the code being open source:

http://www4.cs.fau.de/Projects/JX/

The kernel handles the IO. You just don't need to enter the kernel (syscall) to issue it, it is done via a kernel<->user memory mapped ring buffer. Roughly, I don't really know the details well. See https://lwn.net/Articles/743714/

BTW, you appear to be shadowbanned. I vouched for your post to un-kill it.