Ontario wrongly separated families on results of Motherisk hair-testing program
reason.com
reason.com
That technique is blind testing with published results. 10% of all submissions to the testing lab would be test samples -- a mix of a bunch of known negative samples (from individuals completely unrelated to the crime in question) and a few known positive samples (perhaps from someone already convicted with overwhelming evidence). The lab would not be told which samples were tests and which were "real", and the test samples would be evenly distributed (10% of the submissions from each department; 10% of the "rush" submissions, etc.). Finally, the results reported on these test samples would be made publicly available.
Although this simple, effective, and cheap method would completely transform forensic practice, it will not be implemented. The reason is incentives. The parties involved in forensics are the police or prosecutors and the labs. Both have an incentive to identify the guilty. The accused have a countervailing interest (in correctly determining innocence), but they are not part of the transaction, so their interests are not represented. In fact, the entire system is incented to find people guilty, not to correctly assess people's guilt. So most in the system are actually DISincented to discover and debunk junk science.
Similarly, a rush order is a rush order, but a rush order with a group of people carping on how it's a rush order tends to get more rushed. When the queues are effectively bottomless, stamping 'rush' on an order without the associated pressure seems likely to relegate its status to regular.
Programming was less routinely scientific like that in my own experience earlier in my career when automated testing was less common.
I think the Feynman talk was the one that turned into his posthumous book The meaning of it all, though I'm not sure.
And not only would it let you weed out the pseudoscience but it would let you weed out the labs that are practicing the real science well from the ones that are sloppy (which in my opinion is an equally large problem).
It is a department of the state police, and, obviously enough, is structurally incentivized to make state police performance look good.
https://www.rollingstone.com/culture/news/did-falsified-drug...
Shocking to the blind and deaf maybe. If it's under the executive branch it's a dumping ground for political appointees. That's just how it works in MA. More law, more bureaucracy to enforce it, more dumping, more bad outcomes, more laws, more bureaucracy...
I've lived in three other states and none of them were as bad.
they care about cost, and turn around time. availability to answer their questions, and some level of treating them with basic respect are up there, too (by "basic respect" i mean avoiding things like "i don't have to explain shit to you, you're a dumb cop and i'm the fancy scientist").
if you're regularly telling the DREs that they're wrong, you're going to have some arguing/explaining to do, but it is unlikely to lose you the contract.
the labs that go rogue usually do it for stranger self-motivated reasons than "gotta give the cops the exact results they want"
Yes, these ones are, but the ones that are part of law enforcement offices (particularly, the FBI’s lab) have been even more notorious problems (like the complete junk science of hair comparison) driven by a clear conflict of interest.
so it's a bit harder to get this kind of validation going than just deciding to test the labs.
> ...a mix of a bunch of known negative samples (from individuals completely unrelated to the crime in question)
haaah. you have no idea how many people are on something. getting known blank samples is nontrivial.
> Although this simple, effective, and cheap method would completely transform forensic practice, it will not be implemented. The reason is incentives. The parties involved in forensics are the police or prosecutors and the labs. Both have an incentive to identify the guilty.
there are independent labs that work for prosecution, defense, civil cases, etc etc every form of case you can imagine, and basically throw away all case information for the middle part of the testing process, so the folks who could hypothetically swing things one way or the other don't even know which way they'd be throwing cases if they tampered with it.
the private labs, sure, you could spike (or not) some blood samples and pay for them to be tested.
We're used to hearing about the supreme court restraining the actions of parliament by declaring something unconstitutional but in actuality the parliament exercises far more control over the court than vice versa. Even in the case of the court declaring something unconstitutional parliament could overrule them by changing the constitution itself (although that is extremely unlikely given the political climate).
Besides, this seems like a case of closing the barn door after the horse has escaped. Successfully identifying a problem lab--albeit tardily--is a signal the system is working; not that it needs a complete overhaul. Rather consider what completely unrelated problems would benefit from regulatory oversight at a higher return on investment. Focus on those, not on a solved problem.
25 years is a long time to be improperly ripping apart families. Is there a point at which it becomes a problem? Death tardily solves all your problems after only 80 years, so I assume somewhere between 25 and 80.
And I think the onus is on the suggester to provide data to prove ROI.
That's not what the report [1] actually says. The independent review actually found specific flaws in the way the Motherisk laboratory was performing these tests, such as failure to wash hair samples, non standard hair lengths, misinterpretation of test results, failure to communicate the limits of the tests, and the lack of proper forensic procedures such as adequate records, oversight, and chain of custody.
1: https://www.attorneygeneral.jus.gov.on.ca/english/about/pubs...
The former can be debunked on a case by case basis. The latter needs to be categorically banned from judicial proceedings.
The states who pioneered its adoption (New York and Maryland) have both abandoned it for being costly and ineffective. The FBI has abandoned comparative bullet lead analysis for similar reasons[1]. The National Academy of Sciences found the tests to be "full of holes", and that not enough study was being done to ensure its reliability, which was a finding that the United States President's Council of Advisors on Science and Technology under Obama concurred with.
The general consensus now seems to be this[2]:
> An understanding of the metallurgical principles operative in the melting/casting process as well as the data acquired for this study, indicate that any forensic conclusions which associate unknown bullets with the “same source”, and/or “same box” should fail most or all Daubert criteria.
[1] http://www.washingtonpost.com/wp-dyn/content/article/2007/11...
[2] https://www.sciencedirect.com/science/article/pii/S037907380...
Doesn't this also apply to eyewitness testimony, if not many other forms of evidence?
https://en.wikipedia.org/wiki/Forensic_firearm_examination
>Further criticism came from the 2009 NAS report on the current state of various forensic fields in the United States. The report's section on firearm examination focused on the lack of defined requirements that are necessary in order to determine "matches" between known and unknown striations. The NAS stated that, "sufficient studies have not been done to understand the reliability and repeatability of the methods."[23]:154 Without defined procedures on what is and what isn't considered "sufficient agreement" the report states that forensic firearm examination contains fundamental problems that need to be addressed by the forensic community through a set of repeatable scientific studies that outline standard operating procedures that should be adopted by all firearm examiners.[23]:155 Another report issued in 2016 by the United States President's Council of Advisors on Science and Technology confirmed the NAS's findings, finding only one appropriately designed study that examined the rate of false positives and reliability amongst firearm examiners.[40]
Who cares what those people think about "hair testing".
That's a separate issue because Hair analysis is not inherently worthless. It was just assumed to be vastly more accurate than it actually was.
You can for example tell from a hair sample is not from the same person. IE blond hair vs black hair. You can't do the reverse and tell it is from the same person without DNA testing.
Basically, a different thing had been discredited in the past also involving testing hair. Which IMO is why people though testing hair was junk science vs. the lab not doing their job properly.
That said, I also think testing hair for drugs is so inaccurate to basically be worthless.
Probably you will need some system similar to physics where you can claim something is true untill you have a big confidence in your experiments and the experiments are replicated.
Unfortunately, the people passing judgment (judges, lawyers, juries) are generally not scientists, and therefore are equally likely to believe the guy with the paedo-dowser as the next guy.
Unreliable witnesses are as old as the justice system - “science” gives them a veneer of respectability now, just as it has in the past.
I see two problems, neither of which have an easy solution:
1) scientific literacy is dire, and worsening. Education helps, but when anti-science sentiment runs rife in popular culture and media, as it does now, education is met with resistance. I have been reduced to a gibbering wreck trying to explain to otherwise intelligent people that science isn’t “just an opinion”, and that astrology and astronomy aren’t “just different, equally valid opinions”.
2) the concept of the jury trial is broken. It used to be that you would be tried by your peers - people who likely knew you, had a similar background to you, could empathise and make judgments based on a clear understanding of your world. Now, you’re tried by strangers who have usually been carefully selected to be as hostile as possible without an outright admission of bias. I think there’s a happy medium somewhere between the two, as there are obviously biases in both cases.
A short sarcastic laughter would probably have worked better. If people sense their status is lowering because they believe in this "equally valid opinions" nonsense, they will have a strong incentive to update their belief.
Of course, many will be offended, in which case I recommend turning the sarcastic laugh into a benevolent, condescending smile, with perhaps an appeal to their intelligence to soften the blow. Ideally though, you should spin it as if you were ridiculing a third party. This lets them avoid direct confrontation if they so chose, and you still get to make your point.
In any case, they cannot be taught as long as they subscribe to epistemic relativism. They first have to get rid of it, somehow. Doesn't matter if it's for the wrong reasons. Only then can they be taught how to properly change their mind.
Acting condescendingly will only serve to make people think you are a condescending git.
Then again, the best way to do it is to act as if you were ridiculing a third party. If they get offended, try not to notice, and direct your condescension at those other people you're pretending they're talking about. Give them a way to save face.
Whatever the means, the idea is to stress that epistemic relativism is not socially acceptable. Make them feel that thinking such thoughts has consequences.
> Make them feel that thinking such thoughts has consequences.
I do not condone actually carrying out dire consequences, we've had enough dictatorships to show us how this plays out. I'm talking about manipulating people. A little. Because nothing more innocuous that I know of will do.
I think there's probably an argument to be made about there being benefits to smaller communities, but that argument isn't supported by legal tradition surrounding the word "peer".
So some rules must be created for this experts, what tools they use, what software ...
I think the problem is going to get worse before it gets better.
Polygraph is pretty much that.
But what I got out of reading about these issues too, a lot of forensic science is not science and has its own community different from academic community.
My scenario would involve some startup creating some software that would analyze a brain scan(or worse analyze your internet activity) then it will answer for questions like this"Does this person has potential for being a pedophile? The answer would be something vague like "Strong potential"/ "weak potential" , I chose this example because I noticed that when children are involved you get much more irrational laws and the justice is applyed much stronger with less evidence
Though the blogger doesn't actually say believe the parents, they say believe the mother (well, they say "don't automatically discount the mother" which in context appears to be a weaselly way of saying the same thing) which is a bit more specific. One of the cases seems to involve an accusation from a husband, that the test backed up. So even if the lab was tossing coins to come up with their results, we also have to assume a husband and father lied about his child's mother's behaviour.
This should be treated like a criminal matter, with the same standards of evidence. If we're comfortable letting a criminal go free of prison because guilt can't be proven beyond a reasonable doubt (and we should, for the record), we should be comfortable letting a child stay with its parents until the parents inability to care is established beyond a reasonable doubt. Or at the very least, the parents in question should have a court appointed defence lawyer.
We could start by adding Fathers back into the equation, and not automatically assuming that a child is in danger if they're not in their Mothers' sphere.
I think this is the worst thing that governments have done for families: the idea that a single-parent (Primarily the Mother) family is anything less than ideal for the children needs to attain mainstream acceptance. Dads need to stay, and Mothers need to move over and make way for it.
>According to the report, a married father spends on average 6.5 hours a week taking part in primary child care activities with his children. The married mother spends on average 12.9 hours. Since two-income households are now the norm, not the exception, the above information indicates that not only are mothers working, but they are also doing twice as much child care as fathers.
Fathers stepping in is one way it'll change. Mothers stepping out is another. To find the balance we have to make room for it.
If we're comfortable letting a criminal go free of prison because guilt can't be proven beyond a reasonable doubt (and we should, for the record), we should be comfortable letting a child stay with its parents until the parents inability to care is established beyond a reasonable doubt.
The difference is that in one scenario it’s about punishing something that happened and is done and in the other it’s to prevent ongoing/future harm. If you’ve seen some of the things parents are willing to do to their own kids, you would see why it’s not that simple.
Letting a criminal go free (think serial-murderer, rapist) is just as valid an argument for future harm.
I don't see why you treat these 2 cases differently.
The rights of a child to be abused and the rights of a parent to not have their child be taken away are both very important. It’s my subjective impression that the “child abused” case is much more common than the “child taken away where no abuse occurred” case. I see no reason that our laws should be biased in favor of letting that case occur.
If a child has been living with a mother that (apparently) has 18 drinks a day (getting to 18 drinks a day while still being a functional human is not something you get to overnight, it's a long process) without obviously suffering, this is cause for grave concern, it is a significant risk, but it's not an immediate risk. In this context it's also important to keep in mind that removing a child is possibly the correct answer in many cases, but it hurts the child 100% of the time. That hurt must be less, by a very significant margin, than the hurt of staying put.
Given that we’re talking about a clearly contentious divorce, it’s sadly not entirely unjustified to assume this might happen.
Best case scenario the couple wasn't married very long and doesn't have much in the way of joint possessions and no kids and they both work so you don't have alimony concerns you still need to decide who has to move out.
As such, there is necessarily a tradeoff between the rate of hits (correctly identifying "bad" parents) and false-alarms.
The question then becomes: what's an acceptable false-positive rate?
Even in other applications, you wouldn't just come up with an acceptable false-positive rate. You'd consider that as part of a larger process.
How often do you test? What other tests are available? What's the cost/benefit tradeoffs?
So for example in some medical tests they tell people of certain ages to just come in for tests. They review and change that age based on evidence. Some tests may give borderline results so people get asked to come back sooner than average if they only just pass. Better techniques for confirming or denying the conclusion may be developed for use before doing anything drastic and irreversible.
False-positive rates (of the multiple different tests available) are only one factor.
I would guess almost all of these things are already done in the current case under discussion, e.g. as mentioned before one case involved accusations from a husband. Now maybe that's just two false signals that happen to agree, but already you're talking about two false-positive rates you need to consider. Judges and social workers are probably making snap decisions based on gut instinct and rules of thumb as well that factor in. Presumably a case needs to be pretty serious before removal is even considered.
From just reading a libertarian blog that focusses on goverment intrusion into child-rearing though, you may think people are having their hair snipped randomly in the street and finding their children gone when they get home. A false-positive for that hypothetical case would obviously need to be much, much lower to be acceptable than in tests used as part of a bigger process.
I don't disagree wit you, but none of what you say gets around the fact that once you start removing children from the custody of their parents, a non-zero FA rate sets in.
>Better techniques for confirming or denying the conclusion may be developed for use before doing anything drastic and irreversible.
Case in point: this is also a signal-detection problem. You can certainly try to build a more sensitive detector, but the infamous tradeoff doesn't magically go away.
As such, you have to have at least some idea of what kinds of FA-odds you're wiling to tolerate, else you're just flying blind.
And I must insist on this point: the universe doesn't care how disturbing one finds the Hit/FA trade-off. It's still there, even if one doesn't want to look at it.
Then you cannot, under any circumstances, remove a child from his or her parents.
I mean this in the strictest, most mathematical sense.
Are you sure that's what you want?
They would've known their tests were being used in a court of law. They could've stopped this and they didn't. I hope they face consequences.
For example, you could appeal a traffic ticket decision, but appealing a family court decision could be fairly difficult to impossible task in some jurisdictions.
"As you will learn from reading our interviews, in most states a "temporary" decision made at a 10-minute hearing is generally unappealable and, from a practical point of view, permanent. How reliable is the information that a judge receives? Judge Duncan headlines one section "Lies Usually Go Unpunished."
John Oliver did a special about forensic science. The skit at the end is hilarious.
Nope.
If you get a minute, I would watch this video on child protective services:
But, yes, without actually proving that there was drug abuse, those children should not have been removed.
Translation: Info distribution on the pharmacology of medications on kids. I.e. is it safe to take Advil while breastfeeding? This kind of information changes as the latest studies are done and motherisk has people take studies and translate them to opinions mothers can use.
I've personally used their info and it's been very useful. Their drug testing lab scandal is a travesty and should be condemned, but other offerings of motherisk are scientifically sound. Did you know cough syrup for children is ineffective and shouldn't be given? It's a new thing apparently.
Off-Topic: Why the hell motherisk decided to run a drug testing lab is beyond me. It has nothing to do with its original MO.
i suggest that when thinking about the various forensic "sciences", a good way to think about their reliability is "was this concocted for purely legal reasons, or is this a bunch of normal scientific/medical techniques bent to forensic purposes?" that will quickly let you divide things up into "bullshit" and "not bullshit".
bite marks, tool marks, firearm testing? all concocted to generate convictions; bullshit. nobody uses any of those things for anything except legal cases.
forensic toxicology, drug chemistry, dna? normal science bent to forensic purposes; not bullshit. those were all developed elsewhere, by scientists doing real science, and then later applied to legal cases.
can the folks doing the testing go bad? yes. the best and only way to address that is for the defense in cases to bring in their own independent experts, and do their own testing if enough sample remains.
at least in the US, the best way to make that happen is to fund public defenders better, which solves all sorts of other problems, too.
https://theintercept.com/2016/09/07/white-house-report-concl...
> “Our main finding is that there’s very low-quality scientific evidence for the claim,” says Niels Lynöe, a specialist in general medicine at the Karolinska Institute in Stockholm, and leader of the team, whose report was published last month. “You can’t use these studies to say that whenever you see these changes in the infant brain, the infant has been shaken – it’s not possible according to current knowledge.”
https://www.newscientist.com/article/mg23230994-100-evidence...