Hacker News Reader

Comment by xorvoid | Hacker News Reader

The real fun thing is when the same application is using “select()” and then somewhere else you open like 5000 files. Then you start getting weird crashes and eventually trace it down to the select bitset having a hardcoded max of 4096 entries and no bounds checking! Fun fun fun.

View on HN

I made a CTF challenge based on that lovely feature of select() :D You could use the out-of-bounds bitset memory corruption to flip bits in an RSA public key in a way that made it factorable, generate the corresponding private key, and use that to authenticate.

https://threadreaderapp.com/thread/1723398619313603068.html

Oh that’s clever!

I love how you've made it Eva themed, my respect to you.

Yeah, the man page says:

    WARNING:  select()  can  monitor  only file descriptors numbers that are
       less than FD_SETSIZE (1024)—an unreasonably low limit  for  many  modern
       applications—and  this  limitation will not change.  All modern applica‐
       tions should instead use poll(2) or epoll(7), which do not  suffer  this
       limitation.

You can recompile libc if you want to change the limit, or at least you could in the past.

You don't have to recompile, just do the following (at least on glibc):

  #include <sys/types.h> // pull in initial definition of __FD_SETSIZE
  #undef __FD_SETSIZE
  #define __FD_SETSIZE 32768 // or whatever
  #include <sys/select.h> // won't include the internal <bits/types.h> again

This is a rare case when `-Wsystem-headers` is useful to enable (and these days system headers are usually pretty clean) - it will catch if you accidentally define `__FD_SETSIZE` before the system does.

Note that `select` is still the nicest API in a lot of ways - `poll` wastes space gratuitously, `epoll` requires lots of finicky `modify` syscalls, and `io_uring` is frankly not sane.

That said:

* if you're only dealing with a couple FDs, use `poll`.

* it's not that hard to take a day and think about epoll write buffer management. You need to consider every combination of:

  epoll state is/isn't checking writability (you want to only change this lazily)
  on the previous/current iteration, was there nothing/something in the write buffer?
  prior actual write was would-block/actually-incomplete/spuriously-incomplete/complete
  current actual write ends up would-block/actually-incomplete/spuriously-incomplete/complete

There are many "correct" answers, but I suspect the optimal answer for epoll is something like: initially, write optimistically (and do this before the wait). If you fail to write anything at all, enable the kernel flag. For FDs that you've previously enabled the flag for, if you don't have anything to write this time, disable the flag; otherwise, don't actually write until after the wait (it is guaranteed to return immediately if the write would be allowed, after all, but you'll also get other events that happen to be ready). If you trust your event handlers to return quickly, you can defer any indicated writes until the next wait, otherwise do them before handling events.

You can see why people still use `select`.

Checking other libcs (note that "edit the header" is not that difficult to automate):

  bionic - must edit the header
  dietlibc - must edit the header
  glibc - undocumented but reliable, see the dance in the original post
  klibc - must edit <linux/posix_types.h> (which, note, sabotages glibc)
  MUSL - must edit the header
  newlib - documented in header, just `#define FD_SETSIZE` before you `#include <sys/select.h>`
  uclibc - as glibc (since it's a distant fork). Note that `poll.c` for old uclinux kernels is implemented in terms of `select` with dynamic `fd_set` sizing logic!

  freebsd - properly documented, just `#define FD_SETSIZE` first
  netbsd - properly documented, just `#define FD_SETSIZE` first
  openbsd - documented just in the header now (formerly in the man page too), just `#define FD_SETSIZE` first
  solaris - properly documented, just `#define FD_SETSIZE` first
  macos - properly documented, just `#define FD_SETSIZE` first
  winsock - properly documented, just `#define FD_SETSIZE` first, but note the API is not actually the same

Oh the real fun thing is when the select() is not even in your code! I remember having to integrate a closed-source third-party library vendored by an Australian fin(tech?) company which used select() internally, into a bigger application which really liked to open a lot of file descriptors. Their devs refused to rewrite it to use something more contemporary (it was 2019 iirc!), so we had to improvise.

In the end we came up with a hack to open 4k file descriptors into /dev/null on start, then open the real files and sockets necessary for our app, then close that /dev/null descriptors and initialize the library.

There's no need to actually do all the opening if you control the code.

You can do anything with `fcntl(F_DUPFD{,_CLOEXEC})` and `fdopen`.

If we had control of library code, we'd just get rid of select()...

Though we did use the dup trick in another case!

> trace it down to the select bitset having a hardcoded max of 4096

Did it change? Last time I checked it was 1024 (though it was long time ago).

> and no bounds checking!

_FORTIFY_SOURCE is not set? When I try to pass 1024 to FD_SET and FD_CLR on my (very old) machine I immediately get:

  *** buffer overflow detected ***: ./a.out terminated
  Aborted

(ok, with -O1 and higher)

You’re right. I think it ends up working out to a 4096 page on x86 machines, that’s probably what I remembered.

Yes, _FORTIFY_SOURCE is a fabulous idea. I was just a bit shocked it wasn’t checked without _FORTIFY_SOURCE. If you’re doing FD_SET/FD_CLR, you’re about to make an (expensive) syscall. Why do you care to elide a cheap not-taken branch that’ll save your bacon some day? The overhead is so incredibly negligible.

Anyways, seriously just use poll(). The select() syscall needs to go away for good.

You've had a good chance to really see 4096 descriptions in select() somewhere. The man is misleading because it refers to the stubbornly POSIX compliant glibc wrapper around actual syscall. Any sane modern kernel (Linux; FreeBSD; NT (although select() on NT is a very different beast); well, maybe except macOS, never had a chance to write network code there) supports passing the descriptor sets of arbitrary size to select(). It's mentioned further down in the man, in the BUGS section:

> POSIX allows an implementation to define an upper limit, advertised via the constant FD_SETSIZE, on the range of file descriptors that can be specified in a file descriptor set. The Linux kernel imposes no fixed limit, but the glibc implementation makes fd_set a fixed-size type, with FD_SETSIZE defined as 1024, and the FD_*() macros operating according to that limit.

The code I've had a chance to work with (it had its roots in the 90s-00s, therefore the select()) mostly used 2048 and 4096.

> Anyways, seriously just use poll().

Oh please don't. poll() should be in the same grave as select() really. Either use libev/libuv or go down the rabbit hole of what is the bleeding edge IO multiplexer for your platform (kqueue/epoll/IOCP/io_uring...).

Or back in the days of Solaris 9 and under, 32-bit processes could not have stdio handles with file descriptor numbers larger than 255. Super double plus unfun when you got hit by that. Remember that, u/lukeh?

Reply on news.ycombinator.com