> There are also latency and bandwidth benefits how they setup their RAM just from pure physics
What sort of physics? Dedicated GPUs achieve massive memory bandwidth without needing to put all of their memory on-die.
What sort of physics? Dedicated GPUs achieve massive memory bandwidth without needing to put all of their memory on-die.
But even there, the fastest AI accelerator GPUs are putting memory on die, and using chiplet designs, to get the memory closer and closer to the cores.
Ideally, RAM and compute should be combined. That's kind of what our brains do. We'll probably need more mature memristor technology to achieve that one day.