When Apple releases a MacBook Pro with 64GB of unified memory (assuming they will) — won’t that be amazing for machine learning? I am under the impression that GPU memory is a huge factor in performance. Also, is there any way that the neural engine can accelerate training — or is it just for executing trained models faster?
I think Apple is aiming more at either training small models, or running pre-trained models. For example Photoshop is starting to integrate neural filters, so NN inference performance can be important for some desktop applications.
Comparing raw numbers between vendors is always tricky, but it looks like Apple's "cores" are more like Nvidia's 'Streaming Multiprocessors' (SM's), of which their cards have between 14 and 100. M1 seems to perform similar to their older, mid-end desktop cards (1050 Ti has 6 SM's and M1 matches it in benchmarks).
[1] https://www.apple.com/newsroom/2020/11/apple-unleashes-m1/
So way better than a regular Intel CPU, I've heard comparisons with the GTX 1050. Definitely not going to replace a top of the line GPU for ML.
However, everything so far indicates these will be pretty powerful, as even the M1 is pretty beastly for what it is. So it's possible.