Yep. Google has TPUs, and AMD/Nvidia/Intel/Apple/Qualcomm all have tensor coprocessors now. From a CPU or GPU to an FPGA, the cost/benefit is huge. With every device having tensor cores, not so much. ASIP and ASICs are likely the way to go, at least for common operations like matrix multiplication.