Hacker News Reader

Regarding Tensor cores, it does have them as part of the 32 core Neural Engine. Apple considers AI/ML a consumer feature, all the way down to the iPhone hardware. At the same time, this isn't a data-center supercluster. It's still just a mid sized workstation.

View on HN

The Neural engine is for inference, not training.

There’s no difference between inference and training. Training is simply ~4x more expensive inference.

The submitted article also talks about training models.

There is a difference. We train with large batch sizes these days. The ANE silicon size is tiny and can't do the large matrix multiplications for big LLMs with or without a batch size higher than 1. Meaning that it cannot saturate the RAM bandwidth and that you're better using off the much bigger GPU on the Apple die.

Reply on news.ycombinator.com