Watermarking changes the probability calculations for reasons other than quality. It can't not compromise quality. It literally leads the LLM to occasionally chose different tokens just for watermarking purposes.
Also, as long as the same sampling strategy is used during training as the one used during inference, then the LLM will actually do much better with the biased sampling strategy than it would with a fair one - because that is what it was trained to optimize.
So what? By definition with this system the LLM will chose tokens it otherwise would not, purely for watermarking reasons. Yes this token may have had a decent likelihood of being chosen anyway, but it wouldn't have been chosen and now it was for reasons nothing to do with output quality.
I'm not sure what your last paragraph is trying to say. The blue/green list system changes what output the LLM would otherwise produce. You can't train it to produce watermarked output with this system. If you tried to, there would be no delta between trained output and watermarked output for you to be able to detect.
My second point is that the training of a model by definition maximizes the fitness between the final output function and the training metrics. So, if the model is trained with the watermark applied, the training process will minimize the function `model_error(input) = |watermarked_sampling(model_output(input)) - expected_output(input)|`, by definition. This means that a model trained in this way will perform better when sampled using the watermaked_sampling method than if using, say, top_k sampling.
>My second point is that the training of a model by definition maximizes the fitness between the final output function and the training metrics.
Right, but the fitness in question is watermarked text fitness, not fitness for any user interests aligned metric. You're basically saying that if we train LLMs on watermarked text they'll be really good at producing text that looks watermarked, and then we'll stick an actual watermark on top of that. Screw whatever the user wanted it to be good at.
Yes, that's the goal that was used, but they are quite simplistic and crude methods, not some specifically designed function, with carefully fine tuned parameters or something. So, if a basic function like top_k can improve model utility, it's not impossible to imagine that watermarking could also happen to do so, or at least not have a significant negative effect. So whether the effect is deleterious or not is an empirical question, not something we can assume ahead of time.
> You're basically saying that if we train LLMs on watermarked text they'll be really good at producing text that looks watermarked
No, you're misunderstanding how the training works. If we train the model's output so that it minimizes the error function after the watermark is applied on it, the model will learn how to produce the best output it can given the watermark. It will produce better text that happens to be watermarked, not "more watermarked text". Same as if you train the model on minimizing `top_k_error(input) = |top_k_sampling(model_output(input)) - desired_output(input)|`, the model will learn to produce better output under top_k sampling, not learn to produce output that's "looks more top_k".
For practical purposes it isn't.
We already use an RNG at inference precisely because it leads to higher-quality output. Changing what function is generating our random numbers changes the sequence, not the randomness from the point of view of a user.
Fundamentally the article is railing against --temp > 0.0. He doesn't know what he's talking about.
You can't as a user tell by how much the quality of the output was degraded. True.
>We already use an RNG at inference precisely because it leads to higher-quality output. Changing what function is generating our random numbers changes the sequence, not the randomness from the point of view of a user.
I'm not saying it wasn't random and now it is. I know how these things work. I said that the quality of the system is in the quality of the probabilities. That quality is being degraded.
How is it being degraded exactly? The probability that it picks each option will still be the same, just deterministic based on a seed generated from the text.
LLMs already use PRNGs. This is just changing the source of the seed. And a different seed does not change the "quality" of the random numbers. Even if you are worried that it somehow might, they can just use a cryptographic PRNG, then it is literally guaranteed that the source of the seed will not affect the output in any noticable way.