I'm having a hard time imagining this SIMD replacement except for extremely narrow use cases. Are you suggesting the PIM would have a full blown IO controller and cache subsystem to fetch remote operands?

I assume PIM is only going to work well for chunky streaming over the data within that particular memory module. Something that address and operate on whole rows at once and has minimal buffering between the RAM access ports and the PIM register state.

A lot of SIMD code can be on two array operands, and I expect this PIM approach only works well if both are stored locally in the same memory "local" to the PIM and where it can efficiently interleave at the natural addresisng and access widths. Too much random access or needing "remote" data sounds like the point where PIM fails and you need the elaborate memory IO controllers and caching subsystems of CPUs sitting on top of the distributed memory modules..?