Hacker News Reader

Comment by greggman3 | Hacker News Reader

This sounds to me like seriously bad API design. I'm not saying the low-level functions that allow handling fancy situations aren't important but shouldn't the default be "it just works" and not "you must write 1000s of lines code exactly correctly or else your program will fail and you'll be powned"

This reminds me of Jonathon Blow's rant on similar issues on XBox and PS2/3 where the API, instead of doing the right thing by default required the developer to spend days/weeks using it correctly and having their app rejected over and over and in this particular case there was zero good reason for it not to handle the issues itself rather than pass it all on to the developer.

View on HN

I think interrupting I/O can be tricky for an application to handle. If write(2) didn't have this behavior, it's likely that a common outcome would be frozen processes that you can't quit with ^C.

There either needs to be a notion of partial writes (what the article describes) or a write that gets rolled back if interrupted (probably really hellish to implement ... Maybe impossible for something like a socket where the remote machine may have already seen some packets).

EINTR or SIGPIPE are famously criticized as poor mechanisms for this. However even without that, interrupting a blocked write would remain a tricky problem.

Writing to a file should be transactional. It either succeeds, or gets rolled back. Journaling filesystems implement it as a crash protection mechanism, but an API where you can commit changes without closing, and can roll back explicitly, would be nice.

Writing to a socket is unlike writing to a local file, and cannot offer such guarantees.

Seems a lot easier said than done, and would probably incur some costs for something most programs would be largely uninterested in.

Case study: Windows has a feature called "transactional NTFS" which I think may have grown out of the WinFS experiment. They duplicated a lot of filesystem APIs with "transacted" versions. I think the ntdll APIs were a little cleaner than the win32 wrappers - maybe at that layer you can set a transaction on a handle and issue normal fs calls? Anecdotally I heard it was flakey. I believe it's in a sort of deprecated state today.

Journalling protects metadata writes (file names, dates, sizes, permissions, root block allocations) not file content writes, such that a FS can be brought up quickly after a crash. If you want transactional/atomic file ops, you need to implement it yourself on top of existing mechanisms, like transactional databases have been doing for ages.

> Journalling protects metadata writes […] not file content writes

As I understand it, the ext4 file system can have full-data journaling with the “data=journal” mount option (instead of the default, “data=ordered”, which is as you describe).

are there measurable performance hits that you take with transactionality?

Certainly there should be. Not always large; I suppose creating and sequential writing into a new file should be about as expensive as non-transactional.

But it sounds from the article that write(2) is more complicated than necessary. Why do some errors cause an error return while other errors cause a signal? I would think consistency in how errors are propagated would be better.

The article I remember reading said that to do it right like most other operating systems of the era did requires a lot careful design and work to get it right, so Unix developers punted and left it to the application layer.

You get bagged on for saying this but it's just inexcusably shitty. And that's that.

You are thinking of "Worse is Better". But the take away of that article, even if the authrs would have preferred otheewise, is that paractical considerations, time to market and ease of implementation beat purity and elegance every time.

A lot of it had to do with code that historically does not check for errors.

> interrupting a blocked write would remain a tricky problem.

Would it? I believe kill(pid, SIGTERM) interrupts it all right. Or do you want to do this without terminating the program? But then we are back to square one: the programs have to handle failed writes somehow. How? "Abort, Retry, Ignore"? And then again, most of this stuff happens deep inside libraries, and every library makes different choices in handling such errors, and when you combine them in a single program, you end up with a mess. As always.

It doesn't... or rather, it doesn't unless the kernel can safely unroll the write from it's insides. Famously NFS mounts can get eternally stuck processes when they die because the other end of the file descriptor doesn't exist and there is nothing the kernel can reasonably do about it.

Uh, return EIO? Or EINVAL?

EINVAL would mean the fd is invalid, which isn't true. The application already opened the file and possibly wrote to it, so EINVAL is not a valid error code.

EIO sounds like it would make sense but can only be returned for errors relating to modifying inodes, the EIO that makes sense for NFS to be gone would be occuring when you call close(2) on the fd, or fsync(2) which has very unpredictable behaviour when it returns errors anyways. Either way, NFS doesn't handle inodes, so an EIO cannot be returned.

That's an overly pedantic reading of the Linux manpage, IMHO. Even then, a) any of those error conditions may be extended to cover NFS failure; b) EINVAL's description already says "fd is attached to an object which is unsuitable for writing" — who says that such unsuitability is an immutable property? After all, writing into read-only fds is EBADF, and c) a new error code could be invented, if one is so inclined.

But then again, POSIX merely states that while EIO shall be returned if "a physical I/O error has occurred", it "may also be returned under implementation-defined conditions". Heck, if you squint at the right angle, even ENXIO could be somewhat appropriate. I don't think that extending the meaning of EIO is worse than making unkillable zombie processes, to be fair.

No, when the process is waiting on read/write to a file, it cannot be interrupted. The signal literally will not reach it until the syscall finishes. This is a deficiency of Linux design inherited from Unix and something that is being addressed over time. However it stems from the real requirement that a device can't just have its operation cancelled, once you've asked the HDD to read a sector, it won't respond until it has finished reading. So the process is stuck in the kernel HDD driver which literally cannot be interrupted. When it's a network or other removable storage obviously interrupts should happen at the media level, but the design did not permit it originally. Currently you can kill an fuse process and NFS has bits to allow signals in the middle of a read, but cifs for example doesn't.

Mmm, but don't modern HDDs use DMA+IRQ for data transfer? The disk driver issues the read instruction and then it's done until the interrupt arrives?

Anyhow, what are the peripheral devices that support only synchronous access? I don't believe there are any such ones in the modern PCs, but maybe there are some in mobile devices? Old floppy drives?

Isn't that the same as EINTR with a different name?

And what happens if some bytes have made it to disk, network, or pipe? Don't you want to tell the caller how many, in case they want to recover somehow?

Congrats, you reinvented the status quo.

One improvement would be to separate the error indication from bytes written. Then you could have a write that fails, with nonzero output. Windows' API allows for this, but I don't know how common it is. It would probably be a source of bugs the same way these write(2) corner cases we are talking about are.

I was discussing a different scenario, "NFS mounts can get eternally stuck processes when they die because the other end of the file descriptor doesn't exist and there is nothing the kernel can reasonably do about it" — the kernel absolutely can do a reasonable thing and return from write(2) with whatever error code.

And yes, Windows API's WriteFile does actually return less lpNumberOfBytesWritten than was passed in nNumberOfBytesToWrite, but only when writing to a non-blocking pipe; the disk writes are either fully written or fully failed. Sockets use their own special API that generally follows the write(2) semantics (although there are some hilarious corner cases with internal buffers: e.g., you can make a 1 GB write to a write-ready non-blocking socket and the internal buffer will grow to accomodate it, but further write requests will return EAGAIN).

Oh, absolutely. The Unix system API is what was easy to implement in 1970, not what's sane for developers nowadays.

Don't invoke system calls directly. Use a library that handles all this stuff for you properly. Better yet, use a language that gives you a better interface than C does.

Advice (which I agree with) that conflicts with the author's previous post referenced in the first paragraph.

I see that post as a double standard. Yes, libraries are mostly crap; that's because code is mostly crap. Fixing the file-writing implementation in a library may take more patience and tact than this author has, but it can be done; fixing what dozens of mediocre programmers wrote on their own is completely hopeless. If you want to talk about an attractive nuisance that lets people write code that sort-of works most of the time but's actually fundamentally broken, that's Unix - and C - all over.

> This sounds to me like seriously bad API design

this illustrates the sad irony of unix's genius. All the people who think like Dave Cutler (or whoever) are eager to bring Cutlerian ideas to unix. But unix is where they are bringing these ideas because unix is what won, and unix won because it did not make Cutlerian design choices.

I'm not saying that an API call that did the right thing wouldn't be great for a programmer; just that there are a lot of (simple) things going on in the kernel and tradeoffs for a simple userspace security model and some of this comes with that territory. And neither am I saying that a Cutlerian model is theoretically unsound. But things play out practically and this is how things have played out, and observers along the way discussed it the whole time.

More to say for sure, but please forgive me. My comment is for people who understand what I'm talking about. Explaining to people who don't understand is a useful and noble thing, but this ground has been covered so many times over ("worse is better"), and it's very detailed and historical and somewhat subtle, and I have limited time atm.

I think the post is dramatized a little. Mentioned SA_RESTART should fix the EINTR problem for most synchronous functions and is used for default signals.

"Doing the right thing by default" is a weird concept.

Who's default are we talking about here? The developers desired state of default, or the users?

Wasn't that the point of the whole O_PONIES thing?

Reply on news.ycombinator.com