Hacker News Reader

I have an MS in statistics, teach graduate statistics to econ PhD students (essentially; it's an 'econometrics' class); do research on statistical methods, etc.

I'm really sympathetic to your point. I would not want the guilt or innocence of myself, a family member, a friend, etc depend on the numeracy and statistical literacy of a jury, with the stats explained to them through lawyers (even my own!)

As to the OP, the key sentence for me is:

> ...the fire had been started by a discarded cigarette, ... the other two explanations were even more implausible

Only TWO other possible explanations for the cause of a fire!?!?! Regardless of the word choice, I think the appeals court has a better intuitive understanding of some of the pitfalls of model-based Bayesian statistics than the OP (namely, if you accidentally put zero weight on parts of the prior, you will necessarily put zero weight on those parts of the posterior).

edit: if I'm going to be listing credentials, I should point out that my econ phd is in econometrics; I don't want to give the impression that my MS in stats is a sufficient credential to teach econ phd students.

I do not think I understand your argument. In the case as you describe it the statistics are done incorrectly and cause a potential false conviction.

How is this different then an expert on autopsies incorrectly does the job which leads to a potential false conclusion.

In both case an expert made a mistake or did their job poorly leading to a potentially unfortunate outcome. Why should one be banned and not the other?

The argument is mainly that it's (intuitively) much more common to make a significant error in statistics than autopsy.

Additionally I'd guess most times if you make a significant error during an autopsy, you are aware of it. Whereas it's much more difficult to know whether you made an error in your statistical analysis or not.

When I made that statement I was think about how there have been blood spatter experts, bit mark experts, bug forensics experts that have all been discovered to frauds and cast doubts on to large swaths of forensics techniques that were accepted in general(by the court and public not always other experts in the field).

That was impressions from remember articles where expertise in the above three fields was tested independently and experts were found to lack predictive power. I would have to search for that articles again if I wanted to provide a reference however I am working from memory.

For a criminal trial, with a jury? People are hopeless at statistics.

Even on HN you'll find people making simple mistakes.

Someone confidant presenting a mathematical argument would be persuasive to a jury, even if their numbers are totally bogus.

> In both case an expert made a mistake or did their job poorly leading to a potentially unfortunate outcome. Why should one be banned and not the other?

People doing the job poorly should be banned. But the nurse case is interesting because poor statistics became self-fulfilling.

"We have some deaths, and this nurse is present when they happen, and so they are all suspicious", becomes "all the deaths that happen when this nurse is working are suspicious, and the deaths that don't happen when this nurse is working are not suspicious". And that sloppy thinking gets turned into "probability is 1 in a gajillion that these happen normally by chance."

This is the case with any domain area knowledge though not just math and statistics, it is true for most any expert that gets up on the stand.

That is why both the defense and the prosecutor can bring forth their own expert witnesses.

How is this case different?

Its not. There are strict guidelines about the use and presentation of expert testimony.

>People doing the job poorly should be banned. But the nurse case is interesting because poor statistics became self-fulfilling.

>"We have some deaths, and this nurse is present when they happen, and so they are all suspicious", becomes "all the deaths that happen when this nurse is working are suspicious, and the deaths that don't happen when this nurse is working are not suspicious". And that sloppy thinking gets turned into "probability is 1 in a gajillion that these happen normally by chance."

Investigators have been known to be unduly influenced by pretty much any kind of evidence imaginable and then all further evidence is construed to support the original assumption/assertion. I need a more concrete difference or divider to promote other methods above statistics.

> Investigators have been known to be unduly influenced by pretty much any kind of evidence imaginable and then all further evidence is construed to support the original assumption/assertion.

A different way of phrasing this is that investigators have been known to incorrectly apply even standard tools that they've been explicitly trained in.

> I need a more concrete difference or divider to promote other methods above statistics.

If by "promote" you mean that we should try to increase the statistical literacy of lawyers, police, judges, and the general public, than hells yeah. But encouraging armchair statisticians to try to determine causality and guilt from badly-collected observational data that's analyzed by financially-invested parties...? In the highest stakes setting we have? Really?

I believe the point being made is that it's very difficult to be assured of accuracy when applying numerical estimations to events. There is no discernable difference between the correct answer and an incorrect one, which makes the use of this technique incredibly dangerous when determining the guilt of a person in the court of law.

> There is no discernable difference between the correct answer and an incorrect one,

For non-experts in forensics vs statistics can an average jury member make better decisions about expert level results in forensics compared to statistics/math? It is possible that statistics and math are just less common knowledge and most jury members would then be able to discerne finner differences in forensics then compared to statistics. If this is the case though there is nothing incorrect about the tool, statistis which seems to be implied in the original article, and it is just a lack of common understand that in theory if correct would allow for a new tool to be used.

Would you trust a jury to recognize that the prosecuting attorney got the stats wrong if it was your ass on the line? I would not.

I currently can not distinguish trusting them about stats vs about blood spatter, bit mark, DNA, evidence.

Why should I trust them less for stats? Math does hold a honorary position in society while also being often being disliked(USA here) I will have to think about weather that is enough to single stats out from the others.

Regardless, even if it were "1 in 7 billion"... that ignores the entire possibility that, suppose, someone was intentionally trying to frame her.

In that case, her chance of being present would be 1 in 1. But it doesn't mean she did it.

> In my opinion statistics alone can never be adequate for conviction.

This is in an English court, and it is a civil case. Thus, there is no "conviction". For a criminal case the standard is "beyond reasonable doubt". For a civil case the standard is "balance of probability".

So, no-one is going to jail or getting a criminal record. But they might have to pay compensation to someone else for fire damage.

The phrase "...beyond a reasonable doubt" comes to mind. I would hope a jury involved in such a case would be more reasonable about actual hard evidence, but I hear that juries are selected for emotionality and not rationality. I would hope that the judge in such a case would be more reasonable and demand some actual evidence.

Such is the case in a fearful populace, and one that demands vengeance over justice. I appears as though something bad happened, so someone must pay and ... here's a magic forumla with missing variables that says you might have done it so ... GUILTY.

> I hear that juries are selected for emotionality and not rationality

The case the parent was talking about took place in the Netherlands. I don't think they even use juries in the Netherlands (though I could be wrong about that).

Even in the UK -- the jurisdiction the article is about -- juries are purely random. We don't do the American practice of lawyers from both sides being able to reject jurors for their own reasons (as opposed to an actual good reason like their knowing the defendant, and the judge makes the decision on that).

Correct, no juries in the Netherlands.

wow. Err, what may I ask do you use?

Civil law systems by and large do not use trial by jury. The judge (or judges) pronounces verdict.

The Netherlands is it seems one of the few democracies that do not use juries for any trials. Civil or not. The UK certainly has judge only civil trials (commercial law is basically a way for two barristers and a judge to get paid for sitting and talking for two months)

http://www.dutchnews.nl/news/archives/2008/04/judge_calls_fo...

Most democratic countries use civil law, actually, though I don't think that's particularly relevant. There's a much stronger correlation between language and system of law than there is between form of government and system of law. Note the similarity between [0] and [1], and the complete lack thereof between [1] and [2].

[0] https://en.wikipedia.org/wiki/File:Anglospeak%28800px%29.png

[1] https://en.wikipedia.org/wiki/File:LegalSystemsOfTheWorldMap...

[2] https://en.wikipedia.org/wiki/File:Democracy_Index_2011.png

What common law countries call a "civil case" is not the same was what civil law countries call "a case". It's confusing.

As for the distribution of languages and laws, it's largely due to history. Civil/code jurisdictions show the extent of the former Roman empire and later the influence of Napoleonic Code. Common law countries are almost universally within the Anglosphere and have a common legal heritage commencing in the 1066 Norman Conquest.

Well, it seems to me that many countries do have some sort of a jury system but they use it rarely or not at all. Also, there may be laymen involved (for instance, Sweden and Finland may have two lay judges plus one professional one) but those don't really count as juries.

> the initial Bayesian analysis was plain wrong

The initial analysis used Fisher's Exact Test, which is a frequentist rather than a Bayesian method.

"In my opinion statistics alone can never be adequate for conviction."

I'm not sure I understand what you suggest there, because, what else is there?

Let's say a murder conviction is based on 37 different security recordings clearly showing the defendant murdering the victim. The recordings make it a certainty that the defendant is guilty.

But what is "certainty"? It's an expression of probability, and that comes from statistics. Given the evidence, what are the odds that the defendant is not guilty? Extremely low. We have to come up with pretty crazy alternatives. The probability of the defendant being innocent in this case might be, let's say, 1 in 10^20.

We don't need to actually carry out this calculation, because we don't need the precise probability. It's clear that the probability is extremely low, so we can take a shortcut and refrain from figuring out exactly how low. But that shortcut is still ultimately an exercise in statistics.

You've completely missed the point.

Having something on a security recording can be classed as "hard evidence".

The case in question was one of using circumstantial evidence alone to "statistically" decide whether she was guilty or not.

The law differentiates between circumstantial evidence and hard evidence. People aren't supposed to be convicted on circumstantial evidence alone.

What differentiates hard evidence from circumstantial evidence? It comes down to the probabilities, which in turn comes down to statistics.

Assumptions. The devil is in the details. Throw too much math with too many assumptions at anyone but a panel of PhD's and you will get all kinds of misinterpretation, misunderstanding, and just plain wrongheadedness.

Certainly. My point is that, even if you don't want to discuss it, or use it, it all comes down to statistics and probability in the end. Get the most extreme proof you can imagine for somebody's guilt and that extreme certainty still comes down to probability in the end.

No it doesn't. A blurry possibly-faked nigh-useless photo is still hard evidence.

The value of which is statistical.

What separates hard evidence from circumstantial evidence, then?

a really good lawyer (aka, lots of money).

I probably should have specified a photo of the crime.

Circumstantial evidence is indirect, like you being in the area of something, hard evidence is actually of you committing an alleged act.

Noncircumstantial evidence is evidence which directly supports the point for which it is introduced, without any inference required.

Circumstantial evidence is evidence which requires a logical inference in order to support the point for which it is introduced.

That is roughly how evidence is explained by judges to juries in courtrooms.

For example, the victim's blood on the defendant's shoe is noncircumstantial evidence because it directly supports the point that the defendant was present in the vicinity of the victim's body. However, finding at the crime scene a brown fiber which is same type of fiber used to stich the defendant's very expensive, limited edition gloves, would be circumstantial evidence because you have to logically infer that the fiber found comes from the defendant's gloves. Alternatively, finding gloves that are identical to the defendant's gloves is also circumstantial evidence unless there is other evidence showing conclusively that those particular gloves are the defendants, i.e., footage of the defendant leaving the gloves there or something.

It all requires inference, though. The legal system just pretends that some inference comes "for free", and some doesn't. You never have "the victim's blood on the defendant's shoe", you have blood which you have reason to believe belongs to the victim on shoes you have reason to believe belong to the defendant. The only actual difference between your two examples is that of degree, i.e. the evidence linking the blood to the victim is much stronger than the evidence linking the fiber to the defendant.

What about when the defense agrees that it is in fact the victim's blood? Many things in a trial are stipulated to be true.

Interesting question, but doesn't quite seem to be the difference between hard and circumstantial evidence.

Not by itself, but if you use it to discount negligible uncertainty then you can use it as a method to reach 'direct' aka non-circumstantial evidence.

I don't know. It seems like the defense can stipulate things that are nonetheless circumstantial evidence. Like, if the defendant acknowledges being in the area at the time, the prosecution no longer needs to deal much with evidence that establishes that, but that's still not "direct evidence" of the crime.

However, the main point of this thread of discussion I think was that even things that legally are considered "direct evidence" - say, testimony of an witness to the crime - do nonetheless involve some inference...

Your choice of examples actually supports the statistical nature of the distinction between circumstantial and noncircumstantial evidence. That the blood on the defendant's shoe came from the victim's body is based on a probabilistic test. That the blood on the defendant's shoe arrived through physical co-presence of the shoe and the blood's source body (whether or not it was the victim's) is also only "hard" evidence in the sense that other explanations are relatively improbable.

The difference between the probabilities involved in the above inferences, and the analogous inferences ineluctably underlying any hard evidence, and those involved in circumstantial evidence like your coat fiber, are purely quantitative. In some sense, hard evidence is that which can be accepted "beyond a reasonable doubt", while circumstantial evidence is that which does not pass such a test.

Unfortunately, "beyond a reasonable doubt" escapes precise definition. Yet, courts need to function, so we tacitly agree that sufficiently large quantitative differences are (legally) indistinguishable from qualitative differences.

(EDIT: I'm saying the same thing as mikeash, whose response appeared while I was writing my own)

You guys still aren't getting it; circumstantial evidence is not about the probability that the evidence is what it claims to be, it is about the necessary inference about what the evidence proves if the evidence is what it claims to be.

Noncircumstantial evidence directly supports the point for which it is introduced if the evidence is what it claims to be. The victim's blood on the defendant's shoe directly supports the point that the defendant was in the victim's presence when the victim was dying if it is established that the blood is the victim's blood.

Circumstantial evidence requires an additional inference beyond this step. The fibers are circumstantial because the fiber doesn't prove anything except that the someone wearing those gloves might have been at the crime scene at some point in time. You require the additional inference that the fiber was left at the scene at the time of the crime, which the fiber by itself does not prove but which can be "circumstantially" proved in combination with other evidence. For example, if the defendant, or some other witness, claims that the defendant was never at the scene of the crime at any point in time (specifically, but not limited to, periods prior to the crime), the presence of the rare fibers from his rare gloves is circumstantial evidence that the only time he was at the scene of the crime was when the crime occurred. In other words, the circumstances surrounding the fiber are the proof, not the fiber itself.

Your second paragraph ignores the inference required to go from victim's blood on defendant's shoe to defendant's shoe was in physical proximity to victim's body. It also ignores the inference required to go from blood is measured as matching victim's to blood came from victim's body. It also ignores the inference required to go from blood was deposited onto defendant's shoe directly by victim's body to defendant was wearing shoe at time of blood deposit and thus in presence of victim's still-bleeding corpse.

The (admittedly pedantic) point being made is that proving 'x' never directly supports anything other than the truth of 'x'. Moving from 'x' is true to 'y' is true always requires further inference.

People aren't supposed to be convicted on circumstantial evidence alone.

IANAL but I think this is very wrong.

Obviously the court will prefer strong forensic evidence if it can get it. But when it isn't, defendants can be, and are, convicted on circumstantial evidence, and this is not considered a miscarriage of justice.

If a security recording captures someone whose face can be clearly seen to be that of the defendant, and the recording shows them actually committing the crime, okay, that's pretty hard evidence. Anything short of that -- their face isn't shown clearly, perhaps, or the recording shows them being in the area but doesn't prove they actually committed the crime -- and you start having to talk about probabilities.

Even forensic evidence comes with probabilities attached. It's routine for fingerprint and DNA matches to be qualified with a probability of error. And often the true accuracy is overstated.

Having something on a security recording can be classed as "hard evidence".

The case in question was one of using circumstantial evidence alone to "statistically" decide whether she was guilty or not.

And yet, aren't all types of hard evidence really just probabilistic as well? For example, there is a non-zero probability that the murderer on the security camera is merely a person with an amazing physical similarity to the defendant. If we could develop an accurate mathematical model of human physical features, we could even estimate the probability that we're looking at a double and not the defendant.

Likewise, when a DNA match is used in court, the forensic scientist typically testifies to the probability of a false match. Once again, the evidence is probabilistic.

In both of these cases, the probability of a mistake is low. But it is non-zero, and it can be quanitified--if not with current technology, then at least in principle.

So, you will never get away from deciding cases probabilistically. And if you simply refuse to assign numerical probabilities to anything even when the circumstances readily permit it, you're not increasing your chance of being right, you're just sticking your head in the sand.

>Let's say a murder conviction is based on 37 different security recordings clearly showing the defendant murdering the victim. The recordings make it a certainty that the defendant is guilty.

This is the problem with this approach. Let's say the guilty party is someone who looks like the defendant. Now it doesn't matter how many security cameras there are, they all record what happened: Someone who looks like the defendant commits a murder on camera. The number of recordings doesn't inherently show whether it was the defendant or a double.

Or let's suppose that the evidence has been fabricated by someone trying to frame the defendant. Someone who fabricates one recording to the satisfaction of the jury can generally fabricate 37.

Or perhaps it was the defendant on the video but it wasn't a murder, it was a staged public death in cooperation with the alleged victim who subsequently went into hiding.

All of these things are individually relatively unlikely, but they don't become any more unlikely as a result of additional recordings.

This is one of the reasons why conviction rates are generally so high. Suppose (picking a number out of a hat, since we can't really know) that 60% of the accused who go to trial are actually guilty as a factual matter. But of the ones who aren't guilty, 0.6% are because a double committed the crime, 0.5% are cases where the defendant is being framed by a third party or the police, 0.7% are cases where the defendant could show a lack of guilt but doing so would endanger someone else and the defendant prefers to risk prison over endangering that person, etc., and when you add up all the individually low probabilities it turns out that at least one of them has happened in a quite substantial percentage of cases.

The trouble is that the defendant who goes to trial and claims to have been framed is not believed, because only 0.5% of cases are genuine instances of framed defendants (and juries intuitively understand that), and the prosecutor is too often allowed to infer from this that the remaining 99.5% of defendants are actually guilty, rather than that some quite large percentage of them are not guilty but for some other reasons, each of which seems about equally unlikely in any given case and is thus subject to the same misleading inference. But the alternative, to rigorously consider each of the individually not very likely alternatives, is extraordinarily expensive -- which is why statistics is given the short shrift.

At the end of the day the problem is that proving guilt to a level of certainty that would satisfy us if we were being rigorous about it would also cost too much and be too difficult for prosecutors to prove in most cases to satisfy our political desire to incarcerate "bad people" on a mass scale without breaking the bank any more than we already do. Rigorous, large scale, feasible budget. Pick two.

Statistically means it's just an inductive argument for guilt, but so is being caught on camera. So I suspect that you just don't mean to use statistical at all.

The fact is there's no hard line between good and bad evidence, though I sincerely hope Bayesian equations are never the deciding factor in court cases. Math is great, but "probably guilty" just isn't good enough unless it's incredibly probable. As in, 99% probability seems to be a near certainty, but 75%? 75% is still a strong induction, but that's a 1/4 chance you're convicting an innocent person. So where do you draw the line?

Also remember that Bayesian Probability is still susceptible to bias extremely easy just by determination of how to define the variables. If I were looking to convince a jury, it would be fairly easy to tweak the variables and have a large degree of control over the resulting probability while appearing unbiased (because of Math!). So in this sense, I think that bayesian probability is pseudomathematics. Nobody's questioning the math, but the conclusions are not purely derived from math.

"Statistically means it's just an inductive argument for guilt, but so is being caught on camera. So I suspect that you just don't mean to use statistical at all."

I don't understand what you're saying. The fact that being caught on camera is an inductive argument for guilt is ultimately based on statistics or probability. What is the probability that the person on camera is the defendant?

Where do you draw the line? The American legal system, for criminal cases, draws it at "beyond a reasonable doubt", and provides no numerical guidance for what that means. But there is a line, and not wanting there to be one doesn't make it go away.

It's not the courts decision that's jarring, it's the argumentation.

If the court argued from human unreliability in enumerating options, the judgement would be sensible. But simply arguing from a mistaken understanding of probability makes it look silly.

Bad statistics shouldn't be considered adequate for anything. I have no problem with solid statistics being used assuming they have been properly scrutinized by experts before being used to pass judgement.

So many convictions come based on simple circumstantial evidence without even considering the statistical basis for those circumstances. Many convictions are entirely emotional bias on the part of the jury. Why should allowing statistical analysis to be considered as evidence be considered worse than the status quo?