The only way you'd notice this is they weren't independent, and the most plausible way that happens is if you're re-completing pre-fills (resampling the same hash function against the same log-probs).
The only way you'd notice this is they weren't independent, and the most plausible way that happens is if you're re-completing pre-fills (resampling the same hash function against the same log-probs).
Anyway, there are lots of cases where text carries very little entropy. E.g. boilerplate code, exchanges of pleasantries, well-worn platitudes and jokes, etc. These are sequences of tokens that will be seen across many, many separate outputs. Watermarking here (on the token following the common sequence) would thus be easily detectable and noticed as a claude style. The longer the hashing context though, the lower the amount of pathological cases with low entropy. It would be interesting to understand the exact parametrization better!