Doesn't look like anyone has really answered your question yet.

There are three main reasons the kernel is slow for networking: per-packet dynamic memory allocation, lots of memory copying, and system call overheads.

The first two can be improved by modifying the kernel, and I think people are attempting to do this. The system call overheads arise naturally from having the networking code in the kernel. Basically every time you perform a system call the kernel has to save the userspace context, do the system call, and restore the context. This takes time and is bad for cache locality.

But as others noted, for most people who aren't cloudflare this doesn't really matter.