It's turned on right now. Can you tell a difference? I can't.
How many ways could I write this paragraph and still convey the same idea? Way more than we're aware of. Hundreds? Thousands? Maybe a lot more? The number of semantically similar variants increases exponentially with each word.
I suspect anthropic could turn their fingerprinting up or down if they want. If it were turned way up, claude would use weird phrasing but it would take very little text to tell if something were AI generated. If they turned it down, it would seem imperceptible to humans, but you would need a large sample to determine (with high accuracy) that a passage was AI generated. There's probably a very large middle ground where humans can't tell, and where it doesn't take a large text sample to know (with high probability) that some text was AI generated.
It is quite counterintuitive, but you can hide texts the same size as the original text in imperceptible statistics of a text.
Compared to that feat, hiding a watermark is very easy.
"The restaurant "
Our next set of predictions might be:
[was, had, offers]
So we append the rank/index of the next token (0, 1, or 2) onto the secret. Given a long enough response, that secret becomes unique enough to use as a watermark. This obviously relies on having full deterministic access to the LLM itself, i.e. I don't believe it will be possible for users to derive the fingerprint from text that they've generated, only Anthropic will be able to.
The immediate objection is that this runs the risk of degrading the quality of the response. I think that's totally valid and I'll be curious how Anthropic handles it.
That's my very rough understanding! If someone with more knowledge wants to expand, feel free.
So the watermark words will be biased towards filler and fluff where invisible substitutions are easier, and the content is less (cough...) load-bearing.
The likely outcome is the development of AI watermark strippers which filter out all the twitches and tells that make default AI writing so annoying.
Google seem to have given up on SynthID for text for now, so this is likely a harder problem than it looks. My guess is Anthropic announced this to meet regulatory requirements. But they don't have a robust detector, and I seriously doubt they have a robust system that can survive trivial rewriting by a different model.
This is how I’m feeling right now.
A “watermark” is as an author’s mark. I struggle to understand how a myriad of different texts will produce the same watermark output. How big does a text have to be to generate this sign? What is the false positive rate (where my own authentic prose—gasp—is falsely accused of being AI). How do you “prove” it’s true? Will Anthropic offer some kind of service?
I find ChatGPT to be overly loquacious, and my preference for Claude is the brevity of output. Does this mean I will now have to suffer Claude’s gibbering, too?