Stupid question: why can't we get a syscall to load an entire directory into an array of file descriptors (minus an array of paths to ignore), instead of calling open() on every individual file in that directory? Seems like the simplest solution, no?
Even assume it takes 1us per mode switch, which would be insane, you'd be looking at 0.3s out of the 17s for syscall overhead.
It's not obvious to me where the overhead is, but random seeks are still expensive, even on SSDs.
You could use io_uring but IMO that API is annoying and I remember hitting limitations. One thing you could do with io_uring is using openat (the op not the syscall) with the dir fd (which you get from the syscall) so you can asynchronously open and read files, however, you couldn't open directories for some reason. There's a chance I may be remembering wrong
https://man7.org/linux/man-pages/man3/io_uring_prep_open.3.h...
https://man7.org/linux/man-pages/man2/readdir.2.html
Note that the prep open man page is a (3) page. You could of course construct the SQEs yourself.
You mean like a range of file descriptors you could use if you want to save files in that directory?
Otherwise you can open a dir and pass its fd to openat together with a relative path to a file, to reduce the kernel overhead of resolving absolute paths for each file.
Notice that this avoids `lstat` calls; for symlinks you may still need to do a stat call if you want to stat the target.