memcached (google it) implements this idea (giant in-memory hash table), and scales to multiple machines. used by facebook and lots of other very high volume sites.

it's often also used as an LRU cache on top of a normal database so that you can still get the benefits of persistence and rich queries but use memcached to lower the number of hits to the disk/DB.