I don't see it mentioned here, but the dask library looks like a promising solution,. It has ways to handle to these kinds of large datasets, and efficiently schedule computations that don't fit a numpy model. Worth a look.