I suspect most of the "leafs" are unusable.
An LLM is layers and layers of non-linear transformations. It's hard to say exactly how information is accumulated. You can inspect activations from tokens but it's really not clear how to define what the function is exactly doing. Therefore error is poorly understood.
Certainly it will occur as the generated data exceeds the original, eg after 1-10T tokens.
I think you could also do this faster by moving down the tree in a depth first manner.
Typically I use this for knowledge transfer, style transfer, catastrophic forgetting mitigation, etc and so I don’t go very far. I usually manually review the data samples before using it.