My prediction: this firm will probably try to get removed from the case, rather than open source their shitty code.
Source: I've worked on MATLAB codebases for various genomics research projects in the past.
It’s such a high-level language it’s hard to imagine what the hell they’re doing with all that code. It’s probably mostly useless cruft from GUIDE.
PHB: Hey, how many lines of code do we have? CodeMonkey: You want a high estimate or a low estimate? PHB: High CodeMonkey: Well, including unit tests, comments, whitespace, build scripts, integration test harness... 170k
At least I hope they have enough testing code to be signficant...
If you want a terrible gui-based gui creation interface: labview awaits.
Also just because eval can be compiled doesn't mean it should. It will forever be a security risk and I will not write code that uses it in good conscious. Fortunately, Mathworks has provided good alternatives. My personal favorite has been variable field names. It really opens up a lot of elegant coding.
https://www.mathworks.com/help/matlab/matlab_prog/string-eva...
That isn't necessarily their choice. The prosecutors will make the decision about whether to withdraw the DNA evidence. They probably won't, given that they would need to give the defendant a new trial, which could lead to an accused murderer getting off. A bad look for any prosecutor.
More to the point, if the firm withdraws from any case where their credibility is questioned, what does that say to law enforcement agencies who are thinking about using their software?
[0] https://arstechnica.com/tech-policy/2015/04/fbi-would-rather...
Never met a lawyer before huh?
Jokes aside, prosecutors pushing through cases they know to be unsound isn't exactly uncommon. Many prosecutors are more concerned with their conviction rates than they are in justice, because that's what they are measured and rewarded by.
If a case seems unclear, and you could spend years working on a conviction that will ultimately fall through, that hurts your ability to do justice for more readily winnable cases. You have to spend the time building a case, do all the paperwork, go to trial, etc. That's opportunity cost. So spending that on a case you have 10% chance of winning just isn't a good use of time. Add that to the fact that conviction rate is a metric used to quantify skill, you're rewarded for serving justice successfully. And that then dictates how much money you can get which can help fund enforcing justice.
I believe you're looking at the moral right/wrong, and I don't believe that is the same right/wrong being discussed in terms of how lawyers often choose cases. At the end of the day, lawyers need work and they get that mostly through word of mouth and reputation. You don't really get either of those when you lose cases.
The right thing for them is to put as many criminals behind bars. They review cases and pick ones they can win. They will attack and find unrelated weak points in your character to win. They believe they are doing the right thing and will use whatever they can legally against you. You being innocent and going to court is means someone made a mistake. To confess to a mistake loses you credibility, to confess to an ongoing process mistake could open up other cases where dangerous people could be set free.
Is that your version of the right thing?
Most people want to do the right thing wherein right thing is almost entirely defined by norms and customs of their environment. If the norms and expectations are high ethical and correct standards people will follow them to the degree they are able.
To what degree are such standards broken or defective in America though?
Lest we forget the head lawyer of Texas a state home to aprox 27 million people or around 8% of the nation is a man whose own prosecution has for years only been stymied by the difficulty of prosecuting the man at the head of the states justice department. Either 8 or 9 (I've lost track) directly beneath him have resigned and accused him of corruption.
This isn't even an isolated instance corruption is found in fact all over the united states.
Even when in theory we would like to do the right thing we have a hard time establishing what standards are even real. Look at the fact. For proof of that look no further than the science of hair analysis which the FBI spent decades using to convict the accused before we realized that they were incapable of differentiating dog hair from human hair.
Think of entire people going in to work producing work product about imaginary science they were pretending to do competently and sending people to death row in part because of their fake work product.
https://www.washingtonpost.com/local/crime/fbi-overstated-fo...
The justice system in America is a bad joke that is primarily differentiated from say Cuba in that bribes are paid to your lawyer instead of directly to government officials.
It is just that some think the right thing for themself is to maximize their career progress.
And I would not know in general about state prosecutors, but what I know anecdotally second hand, does not sound good.
Presuming rational actors in this case is missing the general problem with the system: people very easily convince themselves they know the truth despite how the validity of the evidence changes. Whatever it said initially, that must be right - it's misinformation 101. Once a belief is established it is much harder to change.
You would prefer that they not be elected? That they would be appointed by some politician, with the public having no recourse?
The fact is that the public like prosecutors who convict people. That's deeply unfair. But it's also deeply democratic.
I very much agree with you: a government has a monopoly on violence and ultimately we all end up trusting it. Too many checks and balances lead to gridlock. Too few lead to oppression. Much of it ends up being decided on inertia. We do it both ways in different jurisdictions, with successes and failures in both.
The prosecutor isn't unilaterally deciding whether the DNA evidence is valid. There will be a public hearing where both the prosecution and defense show evidence about the validity of the DNA evidence, and a court will rule based on that evidence.
- Not revealing all information they are required to.
- Parallel construction (see above)
- Overcharging, with the goal of making the plea more palatable than the cost/risk of defending multiple absurd charges.
- Lying to you while getting to throw you in jail if you lie to them.
As a result, only 5% of federal cases go to trial.
None of behaviors these are rare. If your understanding of the legal system is based on popular culture, as most people’s is, it is basically law enforcement propaganda that has little relationship to reality.
If you read the article and appellate decision which is linked, it says what I just said:
>On Wednesday, the appellate court sided with the defense [PDF] and sent the case back to a lower court directing the judge to compel Cybergenetics to make the TrueAllele code available to the defense team.
Like I’m actually kinda shocked this is the reality. I would have assumed that DNA evidence would have some blessed methodologies and tools/algorithms, with a strict definition of what constitutes a match or partial match specifically so this wouldn’t happen.
In this case we are also dealing with probabilistic genotyping involving DNA Mixtures with DNA from several individual contributors, and most likely degraded DNA. It is the tool the police can use when other more traditional methods is not possible because of the mixture. That should mean the qualitative value of the DNA evidence is lower, requiring even stronger additional evidence from other sources.
What can happen in the U.S.A. is that one lone man says “I saw the defendant do it.”; the defence attorney can point out that the witness was drunk at the time, that he has motive to lie, that he initially reported another story to the police and only later settled on this story, and what ever else to render him completely unreliable.
The jury can nevertheless return a verdict of guilty, and there are no grounds for appeal then, as it is the power of the jury to decide who is “reliable”, and it is not required to explain it's thought process at all.
What a shocking development that such would result into a criminal justice system where a defendant's race and gender plays such a factor.
It takes only one person in the jury to hang the jury. It's not a majority vote it's a unanimous vote.
Methinks the U.S.A.-man often thinks that bench trials in other countries are done by a single juror; they are not and can range from three to twelve in how many professional jurors are required to reach a unanimous conclusion.
But this is not so much about lay fact finding vis-ǎ-vis trained fact-finding, but the rules of evidence.
Scotland also has jury trials, but does not permit that a man be convicted upon the word of a single witness; there must be further independent, corroborating evidence.
There are many other differences with, for instance, the Dutch system that guarantee a fairer trial. One very big one is that in the Netherlands both the defence and prosecution have one groundless appeal; either side if it not agree with the verdict can demand a fresh new trial with different jurors once. — this obviously reduces flukes of justice.
The other is far stronger rules of evidence and more consistent rulings. Juries are very fickle and legal experts rarely know what verdict they will return based on the evidence they saw before them; whereas with trained jurors, their verdict is often similar with the same evidence given to them.
Indeed, one might argue that the practice of plea bargains, which would be considered unconceivably unethical in most jurisdictions, are actually the saving grace, as they permit stability to this otherwise fickle system as the negotiations between both parties are more reproducible given the same evidence, than fickle juries.
Or of course, that the U.S.A. permits conviction based on the sworn testimony of a single eye witness, which is noteably unreliable.
All of these are black boxes that are routinely meant to convict. — it would not surprise me if such software were far more reliable than human eye witness accounts, but if there's one thing I noticed, it's that a man is seldom afraid of bad matters, he is only afraid of bad matters produced by new technology; far worse matters can stay, so long as they be ancient enough.
It's not like you review all scientific evidence and re-do the experiments that lead up to the discovery of <insert some evidence method> in the first place. Validating all that would also take years and much of it can be established as generally accepted by all parties. Similarly, there will be some trust involved with this source code as well. Getting the opportunity to look for bugs is essential in my opinion, but it needn't take multiple years. Focus on the parts you doubt, similar to what you'd do if you were reviewing the scientific method used in analog evidence.
Of course, the two aren't identical. Validating scientific methods and validating a program is different in that the program is proprietary and the science (usually) merely behind a paywall. The latter can then be replicated by others and becomes established. The former will only ever be seen by that company and doesn't become established. So scrutiny is necessary, but after a couple cases that used an identical version, requiring access without articulating particular doubts would unduly delay the case. It doesn't seem unreasonable to start trusting the program after a bunch of defendants had experts look at it and found no way to cast doubt on its result. If you don't think software of 180k lines can be used in court under such circumstances because it would take too long to review, we should throw out pretty much all software anywhere in the judicial system. (That's not what you said, but some of the replies including yours hint at that.)
Actually, it is. That's how science works and that's how convictions often get overturned.
> Validating all that would also take years
Are you suggesting that unvalidated data is being used to prosecute crimes?
> and much of it can be established as generally accepted by all parties.
The point here is that it isn't established as generally accepted by all parties.
> Similarly, there will be some trust involved with this source code as well.
"Trust but verify"
> If you don't think software of 180k lines can be used in court under such circumstances because it would take too long to review, we should throw out pretty much all software anywhere in the judicial system.
I firmly believe that if the source code isn't available to review by all parties, including the public, then it shouldn't be used in a criminal court.
Yes. Pseudoscience is the bread and butter of criminal forensics.
There are a few important differences between a generally accepted method, and some Matlab black-box that you feed an input into, and it prints out 'guilty' and 'not guilty'.
1. The former is based on centuries of peer review, where the best ideas eventually get selected for. The latter is an externally un-reviewed application, which encapsulates the best of whatever we could ship by Thursday.
2. You can call an expert witness to the stand, and ask them questions about the state of the art of <some evidence based method>. You can ask them why. You can ask them about how certain one should be about their statements. You can't cross-examine a black box.
The actual solution to your quandary is to require that forensic analysis services must pass an annual, independent, double-blind analysis of the accuracy of their methods, before they are used in a courtroom - and that the results of those audits are made available to the defense.
It's one thing for a man in a lab coat to take the microphone and say that their methods are accurate 'to within one in a million'. It's quite another to see an audit, where 100 samples were sent in for analysis over six weeks, and only 92 of them were analysed correctly.
A jury might still convict on the basis of that 92% accuracy, but only if other meaningful evidence points against the defendant.
Unfortunately, the reality of forensic science in 2021 is that most of it is sloppy bunk, with no assurances of accuracy.
Agreed! But if that's the standard, it still doesn't involve letting the defendant see the source code.
Bad stats, especially around DNA, has convicted many innocent people.
BTW, law and Order did an episode on bad DNA science convicting someone.
Or completely fictitious.
Have you heard the story about the FBI crime lab and the “science” of fiber analysis that they developed, and not only used in federal criminal trials but also provided as a service for state and local agencies for decades?
Or the phantom of Heilbronn, where dozens of crimes were linked to a single woman. Who turned out to be the lab technician that assembled the kits. Doubts started once they discovered the caucasian female DNA in cells of the charred remains of a black male.
I often wonder how prosecuters defend against the use of these cases to create doubt.
the company can always retract their product if they want to keep it a trade secret.
The real solution would obviously that the prosecutors cannot enter into evidence the conclusions of any closed source software.
Of course, this only displaces the issue, what of the black box c.p.u.'s whereupon this software ran?