I didn't say it's not valid, or that it shouldn't have been done. I'm just saying "NN still doesn't help them beat iPhone" is not a relevant comment regarding this test. To go with car analogy it's close to an article about an slightly improved design of one part of fuel injection in new ABC and someone commenting "look, an XYZ from last year still can go faster without it".

The test is a valid observation, but is irrelevant to this article, or to the technology.

Why measure something that doesn't show up in a standard user's test? Here are some examples: lower power usage, lower latency of small operations (rather than throughput of large actions), smaller design (saving chip space), etc. And finally - if this is a good technology, Apple can use it too and get even faster.

You should do that, but not only that. If you just test the way the product is being used, you have a million possible reasons why it could be slow. If you just want to know whether slapping a neural network on the chip is a good idea or not, it's best to test that in such a way as to exclude as many confounding factors as possible.

In general: because how a piece of hardware (or set of hardware) is used today isn't necessarily how it's going to be used tomorrow. Faster hardware allows for use-cases that weren't possible prior.