Hacker News Reader

Comment by mananaysiempre | Hacker News Reader

> io_uring would be the first true non-blocking async I/O implementation on Unices.

I would agree with that statement in isolation, except by that standard (no need for a syscall per operation) the first “true” asynchronous I/O API on NT would be I/O ring (2021), a very close copy of io_uring. (Registered I/O, introduced in Windows 8, does not qualify because it only works on sockets.) The original NT API is absolutely in the same class as the FreeBSD and Solaris ones, it’s just that Linux didn’t have a satisfactory one for a long long time.

View on HN

POSIX AIO is not non-blocking async I/O; it can block other threads requesting the resource. IOCP is a true non-blocking async I/O. IOCP also extends to all forms of I/O (file, TCP socket, network, mail slot, pipes, etc.) instead of a particular type.

POSIX AIO has usability problems also outlined in the previously linked thread.

Remember, all I/O in NT is async at the kernel level. It's not a "bolt-on".

IoRing is limited to file reads, unlike io_uring.

POSIX AIO + FreeBSD kqueue or Solaris ports are functionally equivalent to IOCP as far as I can tell.

This describes kqueue:

https://speakerdeck.com/trent/pyparallel-how-we-removed-the-...

I should do an updated version of that deck with io_uring and sans the PyParallel element. I still think it’s a good resource for depicting the differences in I/O between NT & UNIX.

And yeah, IOCP has implicit awareness of concurrency, and can schedule optimal threads to service a port automatically. There hasn’t been a way to do that on UNIX until io_uring.

Yes, please! And if you're interested, RegisteredIO and I assume you'd drop in IoRing.

In a nicely wrapped PDF :-)

Yeah I’d definitely include RegisteredIO and IoRing. When I was interviewing at Microsoft a few years back, I was actually interviewed by the chap that wrote RegisteredIO! Thought that was neat.

Posix AIO is just an interface. Windows also relies on thread pools for some async io (I.e. when reading files when all the data necessary to generate a disk request isn't in cache - good luck writing that as purely asynchronous)

This is all fine, but Window-NT file access is still slow compared with Linux- this shows up in shell scripts. The reason is supposedly that it insists on syncing during close, or maybe waiting for all closed files to sync before allowing the process to terminate. Shouldn't close finality be an optional async event or something?

The reason is due to file system filters, of which Windows Defender is always there. There is a significant delay from Defender when performing CloseFile()[0].

> As I was looking at the raw system calls related to I/O, something immediately popped out: CloseFile() operations were frequently taking 1-5 milliseconds whereas other operations like opening, reading, and writing files only took 1-5 microseconds. That's a 1000x difference!

This is why DevDrive was introduced[1]. You can either have Defender operate in async mode (default) or remove it entirely from the volume at your own risk.

The performance issue isn't related to sync or async I/O.

[0] https://gregoryszorc.com/blog/2015/10/22/append-i/o-performa...

[1] https://devblogs.microsoft.com/visualstudio/devdrive/

Windows FS stack is still _way_ slower than Linux. Filesystem operations have to create IRPs and submit them for execution through a generic mechanism. These IRPs can get filtered and modified in-flight, providing quite a bit of overall flexibility.

In Linux, filesystem paths are super-optimized, with all the filtering (e.g. for SELinux) special-cased if needed.

But even still, Windows also had to cheat to avoid completely cratering the performance, there's a shortcut called "FastIO": https://learn.microsoft.com/en-us/windows-hardware/drivers/i...

I wrote a filesystem for Windows around 25 years ago, and I still remember how I implemented all the required prototypes and everything in Explorer worked. But notepad.exe was just showing me empty data. It took me several days to find a note tucked into MSDN that you need to implement FastIO for memory mapped files to work (which Notepad.exe used).

That's interesting, why would notepad.exe use mmapped files?

Probably to save space when opening large files?

But it simply isn't slower than Linux.

Robert Collins explains that performance is just as good as Linux and the performance loss on Windows is due to file system filters (Defender)[0].

This is what DevDrive intends (and does) fix.

[0] https://youtu.be/qbKGw8MQ0i8?t=1759

The last time I did FS tests was admittedly around 4 years ago, but back then Windows was several times slower on pure FS performance benchmarks (creating/listing/deleting large number of directories and files).

It used to be _much_ slower, like orders of magnitude slower, especially for directories with a large number of files.

You're not seeing "pure" FS performance. You're seeing all of the abstractions between you and the file system.

To get more of the abstractions out of the way, you want DevDrive. And don't use Explorer.exe as a test bed which has shims and hooks and god knows what else.

I was testing the performance using plain C++ code.

These days there is a full native Btrfs implementation for Windows.

https://github.com/maharmstone/btrfs

It's complete enough you can boot and run Windows from Btrfs.

https://lilysthings.org/blog/windows-on-btrfs/

I wonder how many of these limitations affect this, notably performance.

If WSL1 could natively talk to Btrfs without the performance-sapping translation to NTFS, would that resolve the poor Git performance etc?

This probably has a lot to do with the mandatory file locking on Windows - afaik on Windows, the file is the representation of the underlying data of the disk, unlike on Linux, where it's just a reference to the inode, so locking the file on open is necessary. This is why you always get those 'file in use and cannot be deleted' prompts.

This impacts performance particularly when working with a ton of tiny files (like git does).

> Remember, all I/O in NT is async at the kernel level. It's not a "bolt-on".

All I/O in Linux is also async at the kernel level! The problem has always been expressing that asynchronicity to userspace in a sane way.

Filesystem io (and probably more) is not async at the kernel level in Linux. (Just imagine trying to express the complexity of it in continuations or some sort od state machine!) As such io_uring takes the form of a kernel thread pool. Disk block io by contrast is much easier to be fundamentally async since its almost always a case of submitting a request to an HBA and waiting for an interrupt.

Just imagine trying to express the complexity of [a filesystem] in continuations or some sort of state machine!

Arguably asyc/await could help with this; obviously it didn't exist in 1991 when Linux was created but it would be interesting to revisit this topic.

> Arguably asyc/await could help with this; obviously it didn't exist in 1991 when Linux was created

Wouldn't that just consist of I/O operations returning futures and then having an await() block the calling thread until the future is done (i.e. put it on a waitqueue)?

> Just imagine trying to express the complexity of it in continuations or some sort od state machine!

With Rust in the kernel this becomes somewhat possible to conceptualize.

> Just imagine trying to express the complexity of it in continuations or some sort o[f] state machine!

You’d probably want to use either some sort of code generation to do the requisite CPS transform[1] or the Duff’s-device-like preprocessor trick[2], but it’s definitely doable with some compiler support. Not in an existing codebase, though.

(Brought to you by working on a C codebase that does express stuff like this as explicit callbacks and context structures. Ugh.)

[1] https://dx.doi.org/10.1007/s10990-012-9084-5, https://www.irif.fr/~jch/research/cpc-2012.pdf, https://github.com/kerneis/cpc

[2] https://www.chiark.greenend.org.uk/~sgtatham/coroutines.html

> IOCP is a true non-blocking async I/O.

Unfortunately that's only half-true. You can (and will) still get blockage sometimes with IOCP, it depends on a lot of factors, like how loaded the system is, I think. There is absolutely no guarantee that your I/O will actually occur asynchronously, only that you will be notified of its completion asynchronously.

Also, opening a file is also always synchronous, which is quite annoying if you're trying not to block e.g. a UI thread.

The implication of both of these is you still need dedicated I/O threads. I love IOCP as much as anyone, but it does have these flaws, and was very much designed to be used from multiple threads.

The only workaround I'm aware of was User-Mode Scheduling, which effectively notified you as soon as your thread got blocked, but it still required multiple logical threads, and Microsoft removed support for it in Windows 11.

None of the UNIXes have the notion of WriteFile with an OVERLAPPED structure, that’s the key to NT’s asynchronous I/O.

Nor do they have anything like IOCP, where the kernel is aware of the number of threads servicing a completion port, and can make sure you only have as many threads running as there are underlying cores, avoiding context switches. If you write your programs to leverage these facilities (which are very unique to NT), you can max perform your hardware very nicely.

notably the NT equivalent of select(): WaitForSingleObject and WaitForMultipleObjects had a benefit that one select/wait type syscall could be tickled by any of a network, file or the NT equivalent of a pthread signal.

Reply on news.ycombinator.com