> the goal with having this tech on the phone that the phone won't need to communicate with servers to do this sort of processing?

Yeah, that's basically it. In short, the operations listed are pretty computationally intensive on a CPU, but pretty easy to parallelize for the data structures that are commonly used in machine learning (matrices and tensors). I presume that, by creating an API, Google can abstract out these common ML operations to work on the most efficient hardware on the device, whether a GPU or custom ML hardware such as Apple's Neural Engine. In theory, this should make on-device machine learning and inference more power-efficient, and could reduce reliance on server communication, improving privacy and probably making lightweight ML applications feel faster.