Comment by Phemist | Hacker News Reader

Comment by Phemist | Hacker News Reader

Parent Full thread

I am going off the explanation in the declaude page (and related papers). But I see now anthropic mentions Aaronson's distortion-free watermarking.

Random watermarking functions colour the tokens based on (small) contexts and a secret key. Given watermarking functions are randomly chosen every single time they are used (so essentially not deterministically seeded by the context and secret key), then indeed the completion distributions are unchanged. However, they are deterministically chosen, a given string of text will always have the same corresponding watermarking functions. Tokens scored high by the function see an increased probability of being the chosen completion, those scored low see a reduced probability. I dont see it is different from merely talking about it as green/red and the points hold?

For a given short-context (hash function seeding) you do have detectable manipulation of the completion (how they can read the marking). But, because (assuming enough entropy in context) the hashing function is decoupled from the log-probs, the perturbations are independent from the underlying distribution, so you're still sampling from the same distribution quotient some noise.

The only way you'd notice this is they weren't independent, and the most plausible way that happens is if you're re-completing pre-fills (resampling the same hash function against the same log-probs).

Ok. I dont have a good feeling for the actual completion distributions. The noise sounds problematic. I can imagine it relates to the size of the context used for hashing. You want this as long as possible so that the entropy is higher, but you also want it as short as possible, because edits invalidate the hashing for all the tokens of which the edited tokens are part of the hashing context.

Anyway, there are lots of cases where text carries very little entropy. E.g. boilerplate code, exchanges of pleasantries, well-worn platitudes and jokes, etc. These are sequences of tokens that will be seen across many, many separate outputs. Watermarking here (on the token following the common sequence) would thus be easily detectable and noticed as a claude style. The longer the hashing context though, the lower the amount of pathological cases with low entropy. It would be interesting to understand the exact parametrization better!

Reply on news.ycombinator.com