A few months back someone reverse-engineered private ANE APIs and shown some significant performance improvements compared to CoreML and Metal, on both inference and training.

- https://maderix.substack.com/p/inside-the-m4-apple-neural-en...

- https://news.ycombinator.com/item?id=47257931