To distill a neural net the only thing you need is access to the softmax probabilities. You don’t need the weights. This is entirely consistent with the story—-large anounts of data being exfiltrated, not weights but the raw output before sampling.