How Do You Program a Computer with 10 Terabytes of RAM?
highscalability.com
highscalability.com
I've programmed large RAM machines and it's not that hard. In general, it simply let me run programs I couldn't run previously because they allocated too much memory and crashed or swapped/paged too much.
Having a flat memory hierarchy (all RAM has the same cost) makes it dirt simple. NUMA made it significantly harder because you typically had to structure you program's data, threads to schedule on the appropriate cores or processors.
However, I've found over time that unless you absolutely need to hold all your data in RAM, then spending the additional money to get the largest DIMMs and a motherboard with tons of DIMM slots isn't really cost effective. So long as you can partition your problem, that is a preferred solution in nearly all cases. however, increased programmer productivity can often be more cost effective ("just buy more RAM") and I know people who have been upgrading their computers for years and running the exact same algorithms on their in-memory data sets at very high speeds for years.
Depending on your code, some complex problems can occur. For example, the TLB, which speeds virtual/physical address translations, has its own TLB, and if you're screaming all over memory you can blow out the TLB for the TLB. my experience has been the TLB issues go up with larger memory (since you tend to load more data on each node, and access it with sparser patterns).
I haven't looked at DIMM price curves or roadmaps lately, but I assume that 4TB will be $120K in another 3-5 years, then 8TB will be $120K in 10 years.
With all that said, SSDs greatly reduced the issue of needing lots of memory. I've done jobs that would have required too much physical RAM for my budget by configuring linux swap on a fast SSD. It swapped a lot, but the jobs ran :-)
I had a large multidimensional object and I wanted to smooth it.
So, I loaded the whole object in RAM, did an FFT on it, multiplied it by a gaussian, then did an inverse FFT on it.
Another example would be storing very large hashmaps (as a backing store for a sparse multidimensional array).
If you had a very large biosequence database, and you wanted to map next-generation reads against it, you'd load the database into RAM with an mmap call followed by a block read, then subsequent mappings would be very fast.
Effectively, having large amounts of flat access RAM means you don't have to worry about communication (between nodes, or between node and disk).
If just went L1-3 + RAM, it would greatly simplify the job of programmers and programs. (theoretically) no need for virtual memory, buffering / flushing, etc.
If Java just didn't need to allocate each damn small value object (think struct) separately, I think it'd be a lot better with large heaps.
Array of 1000 non-elementary values could be just one allocation, not 1001. That'd also be 1000 times faster to garbage collect.
Still a few years behind other languages, but its coming...
Their approach is to engineer for the worst case, then everything else is easy ^_^. More seriously, their collectors run all the time and GC the whole heap as well as of course nurseries more often.
I'd add that these systems often have NUMA effects in access times, even if it's one big address space, i.e. each chip has better access to the memory directly attached to it than memory on other chips. The mentioned SGI system uses stock Xeon chips, so it's got these sorts of issues I'm pretty sure.
With a 64TB heap, the question isn't "is it better?" but "is pause time O(1)?" I can't recall what the Azul collector does, but I think G1 has some kind of linear behavior that would make it unusable on a heap that big.
I don't want to be Pollyannaish, but it is true that as time goes on, people will keep developing technology for handling large heaps. G1 was not fully supported until Java 7 and will probably be the default in Java 9. Meanwhile there's another low pause collector that Red Hat is working on (http://www.infoworld.com/article/2888189/java/red-hat-shenan...).