Yep, and this is a perfect example of a base rate fallacy situation... even if the scanner is 99.99% accurate, because an even higher percentage of photos are innocent, most matches the scanner will find will be false positives.
Back in the day when I was like 15 and DC++ was still a thing, I used to browse people's shared folders. One day I came across a file called "the paradox of false positive". It was a 1 pager that described how a machine which is 99.9% accurate at identifying terrorists would be completely useless due to this false positive base rate fallacy you're describing.
It really stuck with me throughout the years. It's kind o remarkable how even a 99.9% accurate heuristic is insufficient at scale.
Which begs the question: lets assume the intentions are pure (which we know they're not but lets be generous), what other options are there when 99.9% heuristic is not good enough? how do you design systems when they're guaranteed to fail as they scale up?
edit: and what do you know, I just saw this as I scrolled down on HN https://news.ycombinator.com/item?id=48816959
Whereas if only 0.001% of your population are terrorists then 99 out of 100 alerts are false positives at which point the system is well on its way to being useless.
There is an important difference between scenarios where we care about the relative versus absolute frequency of errors.
> There is an important difference between scenarios where we care about the relative versus absolute frequency of errors.
The context is chat control without probable cause over the whole population of Europe with a low prevalence. My point, and presumably that of OP, is that even a small relative frequency of errors will yield an unsustainably high absolute frequncy of errors.
> This is merely information provided to a human agent.
It will be in theory. In practice the human agent will just forward the decision. A human agent is not sufficient; you need to test only with probable cause for the kind of scenario we're talking about. The exact opposite of "Chat Control 1.0 and 2.0".
P.S.: The comment I originally replied to choose a very convoluted way of saying that the false discovery rate of the test matters for a proper evaluation. Both you and they explain this by throwing numbers without context in combination with slightly inaccurate definitions. I got the definitions mixed up differently, which led to this follow-up.
> even a small relative frequency of errors will yield an unsustainably high absolute frequncy of errors.
That depends entirely on the rate of true positives in the general population and the rate at which the test successfully catches them. If the success rate is reasonably high and the rate of true positives is within one base ten order of magnitude of the rate of false positives then regardless of volume the stream of reports would be expected to prove quite useful.
To put this in concrete terms, if 1 billion messages are scanned, there are 100 violations, 99 of those violations are successfully detected, and there are an additional 1000 false positives reported, then you've got about a 10% hit rate when examining reports. That would provide a genuinely useful starting point.
But it's not at all clear that we can expect numbers like that. Both because the scanners are likely much worse but also because criminals can't reasonably be expected to stick around on conforming platforms in the event that such measures are enacted.
Even if the reports were 100% accurate I'd still be opposed to it on ideological grounds. I don't think pervasive surveillance of that nature is compatible in the long term with a free and democratic system of government.
> Both you and they explain this by throwing numbers without context in combination with slightly inaccurate definitions.
It was my intent to provide reasoning for all the numbers I put forward. They were meant as examples.
As to definitions I wasn't going by anything formal. I tried to spell out exactly what I meant by each term. Apologies if I wasn't entirely clear about that. Regardless, the precise definitions of the terms aren't what matters here. It's the practical end result - what percentage of the alerts are false?
https://en.wikipedia.org/wiki/Positive_and_negative_predicti...
Suppose I invent a device that can detect whether there is a giant invisible dragon living in your house, and it has an accuracy of 99.999%
Now, I use it in your house and it tells me there is an invisible dragon… so what are the chances that there is a dragon in your house?
Based on your statement, it would be 99.999% likely that there is an invisible dragon in your house. However, we actually know that there is a 0% chance there is an invisible dragon, so even with the positive test result we still know there is a 0% chance a dragon is there.
And there is a saying where I grew up: you need a village to raise a kid, I feel like we lost track of that and feel the issues of that now.
Btw, von der leyen is trying to get stuff like this written down as laws since 2009, it got her the nickname Zensursula.
And it fails miserably.
If anything I find GenZ a lot more focused on explicit consent than GenX.
That's a good way to put it
In fact that's why nothing ever gets done to improve things in the EU/west, because we expect perfect outcome in every new change and we want potential risks to be zero before something new is implemented, so it's easier for leaders to just never do anything, never change anything, just sit and maintain the status quo while we go through managed decline complaining things keep slowly getting worse.
1. The cultural factor is rising expectations for children and their parents, costing both time and money;
2. The political/social factor is nanny states and academic institutions that the public expects to not only teach but raise their kids;
3. Technology. Especially the Internet, mobile devices, social media, and short form content. Technology distracts and isolates both kids and parents.
An example of the three factors at work is the all-too-common local news trope of ”Nosy neighbor calls CPS because the family next door lets their kids walk to school. Whole family traumatized as a result.”
I mean, I'll gladly send you 10k years back and have you tell me how it goes.
No it doesn't. That's just needlessly reductionist doomerist take with no argumentation to back that up.
Define failure and success of the system in this context.
Yes it does. If you're of the mindset that drives one to build a nanny state that is. My assumption is the mindset is that any amount of CSA is completely intolerable and that we must try to reach zero cases and there can be no other goal. If that's the framing then parenting alone fails miserably at it.
I think that's a terrible goal, because it leads to the nanny state.
Or is that where you want other people to end up while you peddle propagandist fairytales about failed parenting?
Of course I'm on your side here, there must be a healthy balance, which modern child micromanagement, and zealous survellance in general, are quite an opposite of. Saying this as someone who survived my happy 80-90-s childhood, roaming freely wherever we wanted with my friends, doing kid things many modern parents would be terrified of, like climbing various industrial structures, crashing on our bikes and sleds and whatnot. And still being an exemplary homeboy in the eyes of my parents and neghbours. Sure, some of my friends didn't make it through perfectly unscathed, some broke a bone or two, and I still have that scar from the sled blade, but that's the way of life.
One can't turn the world into a perfectly safe utopia without turning it into complete dystopia in the process. Education and culture is what makes it safe without such side effects.
And Germans and Europeans looked at that and thought the best place for her is leading the EU?!
Remind me again how she got elected in that position?
Because it seems like the entire EU population knew her being infamous for that, except for the few elites who appointed her there via "democratic process" to the head of the EU.
I am not against EU cooperation, mainly in external security and free market economy. But the system we have is not very democratic, and def not very representative of people. They act like demigods, elected by parliament with no real consequences of their actions.
I disagree. That's an executive power position for an entity that lacks sovereignty. Giving it the legitimacy of direct vote is highly problematic.
Start by giving more power to parliament.
Also most of the EU population don’t know her for anything at all. I’d be surprised if more than 50% of Europeans could name her.
[1]: https://www.covid2020.icu/false-positive-false-negative-simu...
https://www.koffellaw.com/blog/google-ai-technology-flags-da...
The corporate liability of such content being found on their cloud is so insanely nuclear, that they're not gonna wait and ask you "hey are those nudes your own kids or are you a pedo?" before they wipe the account with all pics off their servers.
You typically use the Bonferroni correction when making general statements about a statistical relationship. You wouldn't use it for checking if a particular image shows illegal content. If you kept testing with your image classifier, your significance threshold would need to be continuously lowered and you would asymptotically reach zero.
Relevant XKCD: 882
Edit: Yes, in the section 'Statistical Limits of Data Mining' in the book Mining of mMssive Data Sets it's called Bonferroni's principle.
If the scanner is 99.99% accurate, then most classifications will be correct.
So a quota of 0.1% or even less material being detectably criminal sounds realistic (probably not much less, though).