I think that in principle one could tag the training set with "source" tags and express weights as a sum of subweights for each source tag; during backpropagation, the overall weight and subweight for the training sample would be updated, and during inference linear operations would happen on the subweights as well, while nonlinear operations would scale all subweights by the ratio.

This should in principle allow to determine how much each source influenced each output token of the LLM.

The problem is that this multiplies storage and compute time for tagged inference by the number of source tags, so it may be impractical to actually tag single documents or authors, but might be useful for very broad categories like "copyrighted" vs "non-copyrighted", "synthetic" vs "human generated", "photo" vs "drawing" vs "rendering", year range of publication, etc.