Hacker News Reader

Comment by joshbaptiste | Hacker News Reader

Indeed, there has been a large amount of "bypass the kernel" campaigns in the last couple of months. Robert Graham's 2013 Shmoocon talk has a great introduction into the whys and hows of this movement.

https://www.youtube.m/wco?v=D0atch9jdbS6oSI

Facebook had a job posting that showed up on HN for a position to help speed up Linux's networking stack. While I doubt these improvements will surpass the kernel bypassing model, I'm glad a developer has decided to tackle this head on and help overall efficiency.

View on HN

Look no further than Solar Flare's Open Onload[1], Mellanox's VMA[2], Myricom's DBL[3], Intel's DPDK[4], etc to see that kernel bypass tech is used in 10G+ networking all the time.

It is super fast, but also lower latency. Many of the features used in lower latency environments like busy polling have been slowly making their way upstream[5]. I think Linux will get pretty close to the userspace kernel bypass stuff, and this is coming from a guy who works in a low latency environment for $REAL_JOB.

[1] http://www.openonload.org/

[2] http://www.mellanox.com/page/software_vma?mtag=vma

[3] https://www.myricom.com/support/downloads/dbl.html

[4] http://www.intel.com/content/www/us/en/intelligent-systems/i...

[5] https://lwn.net/Articles/540281/

Another great example is snabbswitch, which combines a lot of really great technologies to provide superlative performance:

https://github.com/SnabbCo/snabbswitch/wiki

Hypervisor, Kernel, LuaJIT, Userspace. Amazing!

There is a lot more than bypass the kernel here. You need special dma hardware that will preload cpu cache lines and stuff like that. Maybe even specialized cpu cache that can be pinned

It seems to be pretty uncommon for Linux setups to have, e.g. TOE enabled, but in my humble opinion it is an easy win on performance on 10G networks.

TOE isn't supported under Linux and it was only a win if your TCP stack was slow (and by "your" I mean Windows). TSO is enabled by default in Linux and is indeed an easy win.

here's the fixed link

https://www.youtube.com/watch?v=D09jdbS6oSI

Direct video link: http://www.shmoocon.org/2013/videos/Shmoocon%202013%20-%20C1...

Reply on news.ycombinator.com