Apple's CSAM hash filter removing peoples photos (did it notify the police too?) could be an indicator of how false positives on big scale work out.
A proper watermarking system should be able to have an arbitrarily high accuracy - as many nines as you want. And it should be able to actually report the accuracy of its judgements.
If you're worried about kids being falsely accused of cheating using LLMs, you should be cheering on these developments.
Probability of a false positive is 3 x 10^-5 in one example.
Seems like about 3 collisions per 100 million pictures. If everyone have 1000 pictures that is 3 collisions per 100 000 users.
Raiding 2 people in my city for made up CSAM pictures would be way to high false positive rate.
But you can also make any picture match a CSAM hash by adding picked noise.