NNUE is much smaller than Leela's net, and has a much different architecture that's optimized more for CPU. Additionally, Leela uses Monte Carlo Tree Search and Stockfish uses Alpha/Beta pruning.
Though definitely not directly comparable, dataset of GPT2-xl is 8 million web-pages. What I mean to say is that this is clearly deep learning.
This isn't true. The size of the training data doesn't imply anything about the size of the neural network.
In the case of Stockfish, the NN is quite shallow, and implemented using a custom framework designed to to run fast on CPUs.
See https://news.ycombinator.com/item?id=26746160 for previous commentary on this.
> Though definitely not directly comparable, dataset of GPT2-xl is 8 million web-pages.
This is irrelevant. You can train GPT3 on a smaller dataset, or a smaller model on the same dataset as GPT3.
> What I mean to say is that this is clearly deep learning.
It's been clear that neural network models are superior since Alpha Go. There's not "Deep Learning vs <something else>" anymore because the <something else> isn't competitive and no one is really working on it.
It's just over 82,000 parameters.[1] That's a very shallow, small NN - by comparison something like EfficientNet-B1[2] is 7.8M parameters, and that's considered a small network.
[1] https://www.chessprogramming.org/Stockfish_NNUE#NNUE_Structu...
My point is that having such a huge dataset would not be extremely useful without using a deep neural net (of at least one hidden layer)