Hacker News Reader

Be sure to check PHK's comment (#5)

madvise + mincore syscalls could be used to get the kernel to preload the pages asynchronously.

Not really; because mincore() does not generate events you'd have to poll to find out when the page-in has finished, which seems pretty wasteful. If I was doing disk I/O from an event-driven program I would rather use the Flash approach of using a small number of worker threads to perform blocking I/O. PHK says "I consider this a deficiency in POSIX standards, not in the concept of VM.", but you have to go to production with the kernel you have, not the kernel you wish you had.

I agree you have to make it work with the real and not ideal kernel :) Another thing PHK misses is that single-threaded approach solves the database concurrency problems for free, and people start to use it more - e.g. Stonebraker in his VoltDB.

Polling mincore wouldn't be that bad as it would only happen between the commands and at least the core of the algorithm would be simple: before executing a command, check if it's data are in memory with mincore, if not ask to load them with madvise, and go to the next command.

What does Flash use worker threads for?

Flash used worker threads for any operation that could potentially block, like disk reads.

http://www.usenix.org/event/usenix99/full_papers/pai/pai_htm...

Reply on news.ycombinator.com