I did something along the lines of his suggestion using OpenGL sparse textures (D3D would call them tiled resources) and persistently/coherently mapped buffers with disk i/o being done with memory mapped files. It's a rather crude proof of concept for on-demand loading of large textures (I used a 16k x 8k satellite image). I didn't properly detect "page faults", but I had some of the mechanisms implemented (outputs yellow pixels instead of page faults).

To make it work fully end-to-end, it would look something like this:

    1. Shader samples from a sparse texture, and detects that the requested page is non-resident.
    1b. Fall back to lower mip map level.
    2. Shader uses atomic-or to write to a "page fault" bitmap (one bit per page)
    3. The bitmap is transferred to the CPU
    4. For each set bit, start async copy from disk to DMA buffer (ie. pixel buffer object in GL)
    5. When disk i/o is complete, start texture upload from buffer to a "page pool" texture
    6. When texture upload is complete, re-map the texture page from "page pool" to the actual texture

Now this approach works alright, but there are a number of issues that make it impractical for the time being. Off the top of my head:

    1. Sparse textures are only supported on Nvidia and AMD hardware. Not Intel, ARM or IMG.
    2. Requires Vulkan or D3D12 for step #6 (the demo doesn't do this, so there may be pipeline stalls)
    3. One or two frames of latency that can only be avoided if this was done in the kernel mode driver.
    4. Poor fit for existing KMD architecture (which has its own concept of residency)
    5. Detecting page faults is easy. Detecting which pages can be dropped is hard.

Here's the source code of my demo. It's not pretty because it was a one off demo project for very specific hardware (Android + OpenGL 4.5, which means Nvidia Shield hardware with Maxwell GPUs). The technique is portable, though.

https://github.com/rikusalminen/sparsedemo/blob/master/jni/g...

(In the code above, all the interesting bits are the functions named xfer_*)

Based on the experience from writing this demo, I have to agree with Carmack here. File-backed textures would make a lot of sense for a lot of use cases.