Hacker News Reader

Comment by nothrabannosir | Hacker News Reader

The posix file system API invites these race conditions. but when I want to access the fs from my app, it often feels like the only choice I have is to just accept it.

What can I do to change it? Is there a reliable, widespread and usable successor to the posix API that offers transactions, or "compare and swap"-type operations? Or are we forever caught in this catch 22 between OS writers and application programmers ?

I wouldn't mind an API that can be (unsafely) simulated on regular posix for compatibility, while being forward compatible with the next gen of file IO on systems that do support it, for example.

View on HN

> that offers transactions, or "compare and swap"-type operations?

"Things UNIX can do atomically" https://rcrowley.org/2010/01/06/things-unix-can-do-atomicall...

This depends entirely on what you want to do. You can open files with O_EXCL. You can atomically rename() them. You can use libsqlite3 which provides an abstraction for many operations.

in this situation you can do a stat on the file descriptor after you have opened the file to ensure that it is not a symlink. some of the races with the filesystem can be fixed using the file descriptor family of functions instead of the path family of functions.

How do you get a fd for the link and not the linked file?

There is the non-standard O_NOFOLLOW.

Can you not do an ordinary open without O_NOFOLLOW and then lstat the filename?

Race condition again. The file could be unlinked. fstat is the only safe way of getting stat information for an open file.

Right. The right thing to do is open() with NOFOLLOW, then fstat the resulting fd. Then check for S_ISLNK(st_mode).

Why do you say "non-standard"? O_NOFOLLOW is specified by POSIX.

The Linux manual page is a little misleading, and I didn't dig further into POSIX. Mea culpa.

       O_NOFOLLOW
              If pathname is a symbolic link, then the open fails.  This is a FreeBSD extension, which was added  to  Linux  in  version
              2.1.126.  Symbolic links in earlier components of the pathname will still be followed.  See also O_PATH below.

...

       The O_CLOEXEC, O_DIRECTORY, and O_NOFOLLOW flags are not specified in POSIX.1-2001, but are  specified  in  POSIX.1-2008.

> Is there a reliable, widespread and usable successor to the posix API that offers transactions, or "compare and swap"-type operations?

https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...

the problem is, you can't really avoid this kind of race condition. Not in a pre-emptive system. You can't stat a file and open it at the same time: the whole point of the stat is to see if you SHOULD open it. The best you could do is maybe stat the file after you open it. But I'm not sure what other problems that would cause.

you call fstat on the opened file descriptor and compare device IDs and inodes, as was done in the referenced OpenBSD patch from -96: http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/libexec/mail.lo...

EDIT: corrected year

I could think of multiple ways of fixing this problem changing the API. For instance they could have an api that opens the file only if it is not a synlink.

Like open(O_NOFOLLOW)?

There are many possible solutions:

Obtain a "session ID" through stat and always use that to access the file:

  id := stat("/a")
  read(id)
  write(id, ..) // doesn't protect race access to a file,
                // but at least you're not using the
                // pathname twice, which already solves a
                // lot of problems

Use transactions:

  transid := ftransopen()
  f := stat("/a")
  read("/a")
  write("/a", ..)
  ftransend(transid) // fails if clashes with other fs changes

Have "last modification ID" counters that change for every modification to a file, and allow CAS-style operations:

  mod := stat_mod("/a")
  open_mod("/a", mod)
  read_mod("/a", mod)
  mod2 := write_mod("/a", ..., mod) // fails if /a modified since mod
  mod3 := chown_mod("/a", ..., mod2) // fails if modified since mod2

etc.

All with their own pros and cons, of course, but definitely possible. I'm just curious if there's a specific implementation that's gaining momentum.

okay. neat.

why did you use Go syntax?

What's so strange about stat() on the file after you open it? It's a security check against a known and abused exploit and it's not even hard to do. This seems like a simple fix to the problem.

Just make sure you fstat() the file descriptor and not stat() the file to avoid a second race condition where a malicious actor undoes their attack immediately after you open() the file to hide the evidence.

fstat is a more recent system call than stat, so old code bases do not use it, and mail.local is quite ancient.

Although a quick lookup tells me that it appeared in 4.3BSD-Tahoe (June 1988) and SysVR4 (October 1988), so one would have expected all reasonable distributions to have gotten with the program by now.

A little digging suggests that mail.local appeared in Version 7 Unix from 1979, so it is not a surprise that it doesn't include a syscall invented 9 years later.

Still, that syscall is 28 years old now. It's kind of embarrassing that nobody has gone though and checked for ancient and obvious privilege escalation issues like this. Or I guess they have, but on different OSes. This is one big downside to fragmentation, getting fixes distributed to all of the fragments.

fstat() isn't the complete fix here - it doesn't protect you against opening/creating an unintended file through a symlink, for which you need O_NOFOLLOW (which is a bit more recent).

I had no idea if it was common or not, which is why I said I was unsure.

Reply on news.ycombinator.com