Given how important matrix multiplication with a huge number of fixed parameters is becoming, there is an enormous incentive to design much more efficient architectures where this very simple compute is colocated with memory. Inference cost would come down a lot.