There are ways to do non-blocking disk I/O in *nix (aio/io_submit in linux) but all of which requires you to have an open file descriptor first. Does NT allow you to open a file in an async fashion?
I had not thought about open latency being an issue, that's fascinating. Looking at one of our busy 100G servers w/NVME storage, I see an openat syscall latency of no more than 8ms sampling over a few minute period, with most everything being below 65us. However, the workload is probably different than yours (more longer lived connections, fewer but larger files, fewer opens in general). Eg, we probably don't have nearly the "long tail" issue you do..
That's pretty awesome though, that you have to worry about latency of open().
[1] https://docs.microsoft.com/en-us/windows/desktop/fileio/i-o-...
1. at startup, before threads are spawned, find all static files dirs referenced in the config, and walk them, finding all the files in them, and open handles to all of those files, putting them into a hash-map keyed by path that will then be visible to all spawned threads;
2. in the code for reading a static file, replace the call to open(2) with a look up against the shared file-descriptor from the pool, and then a call to reopen(2) to get a separately seekable userland handle to the same kernel FD (i.e. to make the shared FD into a thread-specific FD, without having to hit the disk or even the VFS logic.)
3. (optionally) add fs-notify logic to discover new files added to the static dirs, and—thread-safely!—open them, and add them to the shared pool.
This assumes there aren't that many static files (say, less than a million.) If there were magnitudes more than that, in-kernel latency of modifying a huge kernel-side FD table might become a problem. At that point, I'd maybe consider simply partitioning the static file set across several Nginx processes on the same machine (similar to partitioned tables living in the same DBMS instance); and then, if even further scaling is needed, distributing those shards on a hash-ring and having a dumb+fast HTTP load-balancer [e.g. HAProxy] hash the requested path and route to those ring-nodes. (But at that point it you're somewhat reinventing what a clustered filesystem like GlusterFS does, so it might make more sense to just make the "TCP load-balancing" part be a regular Nginx LB layer, and then just mount a clustered filesystem to each machine in read-only-indefinite-cache mode. Then you've got a cheap, stateless Nginx layer, and a separate SAN layer for hosting the clustered filesystem, where your SSDs now live.)
No.
As a side note, there was an interesting old hn subthread about async disk i/o of philosophies Windows NT vs Linux:
Don't get me wrong though, caching/file serving in nginx was always very hacky. And blowing latency by blocking on filesystem and yet relying on vfs cache underneath is just one side effect of that.
“hold memory hostage” really doesn't help make your point. It's inflammatory wording one step above spelling it Micro$oft and it makes it sound like you don't understand the engineering trade-offs which the Microsoft engineers made. It'd be much better if you explained _why_ you disagree with their choice and especially whether there are differences in the type of work which you've used it for which might explain that opinion.
Also in my experience it handles slightly more concurrent connections with a fair bit less CPU usage. There are some weird and not very nice parts + limitations to the API that makes it pretty hard to write cross-platform things with it, granted.
This is an old new thing post which demonstrates it is easy tot fall off the happy path on nt. Im not sure the windows world is much better off here.