Hacker News Reader

Comment by catlifeonmars | Hacker News Reader

Is that true for all file abstractions? What happens with NFS?

Think about it, what does it means for a file to be ready? Socket and pipes are a stream abstraction: To be ready it means that there is data to read or space to write.

But for files data is always available to read (unless the file is empty) or write (unless the disk is full). Even if you somehow interpret readiness as the backing pages being loaded in the page cache, files are random access so which pages (ie which specific offset and length) you are interested in can't be expressed via a simple fd based poll-like API (Linux tried to make splice work for this use case, but it didn't work out).

Don’t block devices have a scheduler with a queue under the hood? Couldn’t that queue become full when writing?

(This is a genuine question)

from open(2):

    Note that this flag has no effect for regular files and block devices; that is, I/O operations will (briefly) block when device activity is required, regardless  of  whether O_NONBLOCK  is  set.  Since O_NONBLOCK semantics might eventually be implemented, applications should not depend upon blocking behavior when specifying this flag for regular files and block devices.

I’m pretty sure spinning HDDs can have rather complex controllers that actually try to optimize access at the block level by minimizing the amount the read head needs to travel. So yea there are some buffers in there.

This is one of the things that changed when we moved from SATA to NVME. SATA has only a few "in flight" instructions and NVME does 64k.

Begs a question though: are there any NVME "spinny rust" disks?

> are there any NVME "spinny rust" disks?

My recollection is that NVMe had some features added specifically for hard drives. I don't know if anyone ever bothered making a hard drive that natively used NVMe over PCIe; the main goal was to enable NVMe over Fabrics to work with both solid state and spinning rust drives, so that it could fully replace iSCSI.

I think you’re correct. Your file descriptor may represent an end of a pipe, which in turn is backed by a buffer of limited size. Ruby’s I/O API specifically warns that reading lop-sidedly from e.g. stdout and stderr without `select`ing is dangerous [0].

I’ve experienced deadlocks in well-known programs, because developers who were unaware of this issue did a synchronous round-robin loop over stdout and stderr. [1]

[0]: https://docs.ruby-lang.org/en/master/Open3.html#method-c-pop...

[1]: https://github.com/Homebrew/homebrew-cask/pull/21665

Reply on news.ycombinator.com