For instance, if you are interested in Bayes Theorem like a lot of rationalists say they are, you could talk about the medical test which is 99.99% accurate but for which 90% of the positives are false positives.
https://www.mun.ca/biology/scarr/4250_Bayes_Theorem.html
Imagine that a driver gets hit by accident. He's tested as part of company policy, and tests positive. He gets fired, even though the test only really tells us there's a 33.2% chance he was actually using the drug.
Real world drug tests are a lot worse than 1% false positive and false negative rate.
Every time someone gets fired for a positive test, or loses custody of their kid, or so on, it reinforces whatever statistics are being collected as if the test were a ground truth. They're hardly ever questioned, and there's usually no recourse without an expensive legal fight.
The false positive rate for drug dogs is higher than 40%, for contrast. When a dog "alerts" its worse than a flip of a coin. All that matters is if an officer feels like fucking up your day.
Testing used in situations that are legally significant in people's lives should be required to reach a statistically valid threshold of accuracy, like 99.999% of the times this process is performed, it matches reality. A high sensitivity and high specificity aren't enough, but they're framed as highly accurate and reliable by often well intentioned people who simply aren't thinking in a Bayesian way.
This is what most people don't seem to get. Devices like the ADE 651 or the GT200 were bought by the thousands by law enforcement agencies worldwide, not because they were stupid, but instead, so they could have another "data point" against you that they can use at their discretion.
"Sorry, this dot blinked three times so I'm gonna have to detain you: It's standard procedure, I'm only doing my job."
Antonin Scalia (in)famously commented in one of the Supreme Court's dog-sniff 4th Amendment cases that obviously the police would want dogs that didn't produce false positive alerts, since they wouldn't want to waste their time searching where there were no drugs. The resulting caselaw sets up a situation where a dog can be wrong over half the time and still be used.
The concept that "probable cause on four legs" would be used simply in order to get to search where they otherwise couldn't was apparently unthinkable.
The flawless logic of our leaders is astonishing.
Tend to disagree. It's easy to dismiss one example as "well, medicine is special because XYZ." Multiple examples are the core aspect of showing a general pattern.
He could probably have stopped at 3, 4, or 5 though, not 7.