It looks like training is on CPU too: https://github.com/nodchip/Stockfish/blob/master/src/nnue/ev....