Ah, cool. So is the goal with having this tech on the phone that the phone won't need to communicate with servers to do this sort of processing? I'm struggling to think of a use case for this.
Yeah, that's basically it. In short, the operations listed are pretty computationally intensive on a CPU, but pretty easy to parallelize for the data structures that are commonly used in machine learning (matrices and tensors). I presume that, by creating an API, Google can abstract out these common ML operations to work on the most efficient hardware on the device, whether a GPU or custom ML hardware such as Apple's Neural Engine. In theory, this should make on-device machine learning and inference more power-efficient, and could reduce reliance on server communication, improving privacy and probably making lightweight ML applications feel faster.
I think it's speculation still, but it seems likely that the Pixel Visual Core chip included in Pixel 2s could be a target for the API.
https://arstechnica.com/gadgets/2017/10/google-launches-the-...
The chip is already a target for some ML APIs.
https://blog.google/products/pixel/pixel-visual-core-image-p...
Or maybe on device speech recognition in the future