Hacker News Reader

Comment by majke | Hacker News Reader

Ok, I understand why you might "offload" read() from disk backed files file descriptors. But why "sendfile"? I thought sendfile is properly async in linux... Unless I'm getting something wrong.

"aio_write" -> wasn't the idea of AIO operations to actually be truly async? Maybe it's because it requires XFS. Anyway, I'm confused.

View on HN

Sendfile isn't async, it just avoids the user space copy, and in some cases the copy from the fd to the socket. Aio doesn't require xfs, just O_DIRECT, which can be tricky to get right and more work than userspace guys want to deal with.

From the article

"Although Linux provides a kind of asynchronous interface for reading files, it has a couple of significant drawbacks. One of them is alignment requirements for file access and buffers, but NGINX handles that well. But the second problem is worse. The asynchronous interface requires the O_DIRECT flag to be set on the file descriptor, which means that any access to the file will bypass the cache in memory and increase load on the hard disks. That definitely doesn’t make it optimal for many cases."

From what I can see, sendfile isn't really async, it still returns the number of bytes written or error status, and nginx probably needs to wait for that to e.g. log the results, handle short writes, or proceed with whatever next operation must be done to the socket (like closing it). No idea why aio_write needs offloading, though.

They say why in the article

"The asynchronous interface requires the O_DIRECT flag to be set on the file descriptor, which means that any access to the file will bypass the cache in memory and increase load on the hard disks"

They even go into how no interface has been surfaced yet that allows determining whether a file is in cache or not. Also how FreeBSD's aio interface doesn't have the same limitations

I never tried, but mmap and mincore could work (or not, as one of the many special cases in this area). I expect that's not a viable approach for this application, anyway :)

No, sendfile is not necessarily non-blocking (note the difference between async and non-blocking IO), it still blocks the caller if the reading side is a disk device. The advantage of using sendfile is that you need to copy the data from the kernel-space (source) to the user-space (program) and then back to the kernel-space.

Reply on news.ycombinator.com