Hacker News Reader

Comment by smegel | Hacker News Reader

> OS paging is blocking as hell

No, Linux is rubbish. Seriously. FreeBSD does this properly.

Edit: FreeBSD, Windows, OSX, Solaris, AIX, HP-UX(?)...

The way SmartOS performs under memory pressure is completely different from Linux, the OS is still usable where Linux would be completely frozen. Admittedly I don't know the underlying implementation behind this feature.

Do you have any articles that talk about how FreeBSD does memory management differently?

https://people.freebsd.org/~jlemon/papers/kqueue.pdf

Admittedly I didn't read it in detail, but I don't see anything about page-ins there. Can you explain the connection between kqueue and paging?

There is no connection.

FreeBSD does page faults just like everybody else. Your process blocks until the memory is read in.

Sure it would be my pleasure. Kqueue allows a read request to be scheduled that is non-blocking on a page fault. Linux always blocks the thread executing read() on a page fault. This is still true using aio_read(), as all that does is run another thread to call read() which blocks. Which is great for small numbers of read requests but scales poorly.

And the bit from the paper that is relevant:

> A non kqueue-aware application using the asynchronous I/O (aio) facility starts an I/O request by issuing aio read() or aio write() The request then proceeds independently of the application, which must call aio error() repeatedly to check whether the request has completed, and then eventually call aio return() to collect the completion status of the request. The AIO filter replaces this polling model by allowing the user to register the aio request with a specified kqueue at the time the I/O request is issued, and an event is returned under the same conditions when aio error() would successfully return. This allows the application to issue an aio read() call, proceed with the main event loop, and then call aio return() when the kevent corresponding to the aio is returned from the kqueue, saving several system calls in the process.

Linux's family of async I/O functions is io_submit and friends, not aio_read. That paper is too old to be relevant.

Also, using a worker pool for I/O scales just fine and has no real disadvantages when talking to a disk or SSD.

OK, so you're talking about AIO and other people here are talking about mmap. If you have working AIO then you can indeed write a fully async server at the cost of extra memory copies.

Sadly mmap is also blocking on page faults on Linux :(

I would be very interested in how you envision a system that does NOT block on page faults.

Read and write system calls have nothing to do with jump instructions causing a page fault by traversing data structures. That is what Redis is all about - in memory data structures available to clients.

What else can you do other than blocking until the page has been loaded? How would it be possible to resume a single-threaded process while the memory it's trying to access is not available?

You say "hey process, this data is not available yet, but if you listen on this event port I will sent you a notification when it is ready, then you can do whatever you want with it. In the meantime, continue to serve up data that is in memory".

Pretty simple stuff really.

You might be able to do that in response to a system call asking whether a memory page is available, but not in response to a page fault. A page fault occurs if the process tries to access a virtual memory page that's not currently mapped to physical memory. Unless I'm missing something, the only reasonable way of resuming the process at the machine code instruction that caused the page fault is to first make sure the memory is available. Otherwise you will simply get another page fault.

You don't necessarily need to resume the process at the machine code instruction that caused the page fault.

Right, you also have the option of not resuming it at all.

I mean in principle, you could resume in another place. That's what's being done if you register a handler for SIGSEGV, for instance. Not that there's much you can do there, with existing programming models.

1. thread: hey kernel, I would like to read this page of memory.

2. kernel: hey thread, that page is still on SLOOOOW spinning disk, why don't you go off and do something else while I get it for you. I'll let you know with an event notification, so be sure to check in with kqueue regularly.

3. thread: OK then, i'll go off and serve these other people while you do that for me. kthanx.

4. kernel: hey DMA controller, I'd like you to get sectors 4, 5 and 6 from platter 3 on HDD 2 and load them into memory address 0x4fe6bb. Please send me an interrupt when done.

5. DMA controller: OK servo, please adjust read head to this offset. Read head, read me those bytes. Memory controller, please store bytes at address 0x4fe6bb. Hey CPU, here's an interrupt to store in your interrupt table, please wake up the kernel guy and let him know.

6. kernel: wow I just got interrupted. The interrupt seems to map to a request for data from this particular thread. Better let him know. (sends event up thread's kqueue).

7. thread: hey, I just got a kqueue notification that the file is now ready to be read. That means it must be in memory...cool!

Make sense now?

I get that you can do this in response to a system call (the one issued in step 1). You could avoid blocking in Linux as well using madvice() and mincore(), as pointed out in the comments to the article. However, once you get an actual page fault, the only option is to block the process.

> I get that you can do this in response to a system call (the one issued in step 1). You could avoid blocking in Linux as well using madvice() and mincore()

Yes, but the one thing they don't give you is notification the data is in memory. Do you really want to spin a hot loop calling mincore?

> However, once you get an actual page fault, the only option is to block the process.

See steps 1-3 again.

The only way to receive an event would be from a signal, like SIGSEGV. There is no way to hook a jump instruction up to kqueue. In any case, this would mean writing specific code to handle the case, implying it is not simply a difference of the FreeBSD kernel vs the Linux kernel.

This is roughly what 'scheduler activations' and 'kernel scheduled entities' did. Except rather than just being in response to page faults, it was any interaction with the kernel that would block.

Please tell me how it's possible to do something like socket.write(data), when data has been swapped out, without blocking.

1. make a non-blocking read on the data

2. while it is being paged into memory, <DO OTHER STUFF>

3. now your data is in memory and you can update it (writes are async by default even on Linux, as the write just goes into memory and will get synced out by the kernels page sync mechanism but you can override that by setting the O_SYNC flag).

Triggering a non-blocking read and continuing execution requires either writing different code or an extremely elaborate page fault feature that can dynamically rewrite your execution flow behind your back.

Writing different code is not the question I asked. That's not an OS feature. Do the OSes you like have the latter ability, or are you blowing smoke?

Windows can do this - its "non-blocking" read calls are better described as "asynchronous", and take ownership of the buffer until the job is cancelled or it runs to completion. So if the buffer is paged out, that's fine; the program can carry on and the buffer can get paged in when the system needs it (perhaps on an otherwise idle CPU - so it needn't necessarily take any time from the perspective of the calling thread).

The POSIX semantics on the other hand are simply that the read won't block due to lack of input, so it does as much as it can straight away and then returns. If there's data, but the buffer is paged out, the non-blocking read call has to take more time, because the problem is a paged-out buffer and not lack of input.

(The Windows equivalents of man page section 1 are so awful that most POSIX fans just run a mile and install cygwin. More fool them; once you get to man page section 2, it's a lot better. MAXIMUM_WAIT_OBJECTS is lame, though.)

> Triggering a non-blocking read and continuing execution requires either writing different code

Well you are asking about doing something non-blocking. That means you must be in some kind of event-loop scenario, otherwise you would be happy with synchronous operations. And yes, you would need to add a read event on the file, and then trigger the socket.send() once the data is in memory.

I never said it was free. But at least it is viable.

Your response to paging being blocking was to blame Linux. Every OS has paging be blocking. Linux has ways to do non-blocking IO. There is no reason to blame Linux.

This assumes you are doing read and write system calls, not following pointers in memory.

In the Redis article, it is talking about in-memory data structures. The page fault happens from following a pointer to another location in memory, for example, a linked list, or a skip list. There is no read call to replace with an async read call. Your code evaluates the pointer to a page that has been swapped out, a page fault occurs, and the OS has to swap it back in for you to be able to read that memory.

You could potentially create a new signal, for page faults (maybe there is one I've never heard of), but that would still not let you continue executing from the previous location.

Your step 2 is magical, in the sense that OTHER STUFF wants THEIR data in memory as well.

Reply on news.ycombinator.com