Hacker News Reader

Comment by Bombinator | Hacker News Reader

You can set the buffering mode of any file stream with setvbuf. For example, setvbuf(stderr, NULL, _IOLBF, BUFSIZ) sets stderr to line buffered I/O.

View on HN

that may help, but if a write writes more than PIPE_BUF bytes, it isn't guaranteed atomic by the kernel. similarly, stdioing a line of more than BUFSIZ may result in multiple write calls. i don't think posix makes any guarantees there (this is just an empirically based speculation) and i'm fairly sure the c standard doesn't

Don't confuse the C stderr (which is of type FILE) from posix STDERR_FILENO file descriptor (i.e. 2). FILE (in POSIX, and in C since C11) guarantees that each I/O operation is thread safe (and flockfile in POSIX can be used to make larger operations atomic). A low level POSIX file descriptor is not thread safe (although of course the kernel will protect its own integrity). BUFSIZ only matter when writing to a pipe from distinct file descriptors.

i think the previous discussion may not have been clear enough, because you seem to be discussing a totally different scenario

given this program

    #include <string.h>
    #include <stdio.h>

    char large[16385];

    int main()
    {
      printf("BUFSIZ is %d\n", BUFSIZ);
      memset(large, 'A', sizeof(large));
      large[sizeof(large) - 1] = '\0';
      fprintf(stderr, "%s\n", large);
      return 0;
    }

compiled with `gcc -static` against glibc 2.36-9+deb12u7, we get this strace

    execve("./a.out", ["./a.out"], 0x7fffafcb4a30 /* 49 vars */) = 0
    brk(NULL)                               = 0x1e28000
    brk(0x1e28d00)                          = 0x1e28d00
    arch_prctl(ARCH_SET_FS, 0x1e28380)      = 0
    set_tid_address(0x1e28650)              = 1501924
    set_robust_list(0x1e28660, 24)          = 0
    rseq(0x1e28ca0, 0x20, 0, 0x53053053)    = 0
    prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=9788*1024, rlim_max=RLIM64_INFINITY}) = 0
    readlink("/proc/self/exe", "<censored>", 4096) = 21
    getrandom("<censored>", 8, GRND_NONBLOCK) = 8
    brk(NULL)                               = 0x1e28d00
    brk(0x1e49d00)                          = 0x1e49d00
    brk(0x1e4a000)                          = 0x1e4a000
    mprotect(0x4a0000, 16384, PROT_READ)    = 0
    newfstatat(1, "", {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0x7), ...}, AT_EMPTY_PATH) = 0
    write(1, "BUFSIZ is 8192\n", 15)        = 15
    write(2, "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"..., 8192) = 8192
    write(2, "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"..., 8192) = 8192
    write(2, "\n", 1)                       = 1
    exit_group(0)                           = ?
    +++ exited with 0 +++

you can see that the single fprintf call resulted in three separate calls to write(2), even though it is only a single line of desperate screaming. those three calls happen at three separate times, typically on the order of tens of microseconds apart. if that file descriptor is open to, for example, a terminal or pipe or logfile that some other process is also writing to, that other process can write other data during those tens of microseconds, resulting in the intercalation of that other data in the middle of the screaming

threads are completely irrelevant here, except that i guess in an exotic scenario the 'other process' that is writing to the file could conceivably be a different thread in the same process? that would make your remarks about 'distinct file descriptors' and thread safety make sense. but we were talking about entirely separate processes writing to the file, since that's the usual case on unix, and in that case no form of thread-safety is worth squat; what matters is the semantics of the system calls

i don't think posix makes any guarantees about how many calls to write(2)† a call to fprintf(3) will result in, though i haven't actually looked, and i don't think wg14 concerns itself with environment-dependent questions like this at all

______

† or writev(2)

What he's saying is that as long as you don't mix usage of stdio and raw write(2), you won't have any interleaving problem; because there's a lock, which is why _unlocked variants exist.

that is correct in its own sphere of applicability, but incorrect in the scenario i was discussing, with entirely separate processes writing to the file, since that's the usual case on unix, because each process has a separate lock

(amelius, however, did mention the possibility of multiple threads!)

it also wasn't what they were saying

this thread is starting to remind me of the 'i'm not your buddy, pal' cascades from reddit

Sorry, I meant specifically threads, so the atomicity is purely process local of course. There is an I ternal (recursive) mutex inside FILE.

agreed (well, i haven't looked at how glibc implements the thread safety requirement, but i imagine you're correct)

Reply on news.ycombinator.com