For static file serving, Nginx can bypass BSD socket overhead with the sendfile (or sendfile-like) syscall, though it is disabled by default. On FreeBSD, the sendfile syscall accepts not only the fd to pipe, but can additionally be passed a header/footer such that you don't need to call write(2) at all, though this is not true on either Linux or Windows (AFAIK).

The whitepaper's macrobenchmarks make no mention of sendfile; I presume their tests with Nginx run without sendfile enabled (since it would defeat the point of the benchmark by nature of it skipping all of the write(2) calls). The paper does not appear to detail the configuration used in their benchmarks. I believe that the performance benefit of Nginx+MegaPipe, when compared to an Nginx server configured to use sendfile, is much, much less than +75% throughput.

That said, sendfile has very narrow uses (it's effectively only for sending static files) -- I wish the paper benchmarked Nginx against Nginx+MegaPipe as a reverse proxy rather than a static file server.