Accelerating Compute by Cramming It into Memory

In Big Data (eg Hadoop) and LLM’s, one of the issues is moving lots of data between the CPU and RAM. That makes it slower, increases latency, and uses more energy. We’d rather move less data.

One approach to that is putting computation in RAM. Several architectures have done that. I posted an example. I’ll add a few more below. I’m curious what AI experts think about using these architectures for inference or training.

https://en.m.wikipedia.org/wiki/Berkeley_IRAM_project

https://www.eecg.utoronto.ca/~stumm/Papers/Elliott-IEEEDTC99...

http://venraytechnology.com/Strategy_Papers.htm

https://parallel.princeton.edu/papers/micro19-gao.pdf