Hacker News Reader

I agree that simply disregarding things because they're impossible is not great science, but in this case this isn't great science from the start, because we're simply measuring two of many, many variables. We're only looking at judgement outcomes, and time of day.

The thing is, this result is about the story... about the narrative the findings tell. The original findings told the story of "hungry judges are grumpy judges", but that was conflating correlation with causation. There are other variables which were NOT controlled for (i.e. not great science) that tell a DIFFERENT story.

This article is about the author pointing out that there are multiple ways to tell this story, and this particular way is about as impossible as it gets, statistically speaking.

> This article is about the author pointing out that there are multiple ways to tell this story, and this particular way is about as impossible as it gets, statistically speaking.

The author doesn't cite any statistical examples for his refutation of the study. He simply says that the effect is too big, gives some examples of how big the effect is in other situations, and cites absolutely zero statistics to support his refutation.

If there was a study where they compared relative conviction rates of judges who had a 10 minute snack recess every two hours, if that rate was the same, that would be some evidence.

In fact, it seems as though the conviction rate goes back to normal every time that the judge takes a break, which might suggest that taking a break is the triggering thing, not the food. Would be interesting to see what would happens if the judge were allowed breaks, but not allowed to eat or drink during the breaks.

As a layman, it felt like his article was addressed at professional psychologists, not laymen like myself (or, presumably, yourself).

It read like it was assumed that you'd realize how massively overblown such a result should be interpreted.

It would be like if you wrote a couple tests and your tooling said you had 100% code coverage, you be like "Wait, that just can't be right. I was just writing tests for one function..." Or imagine you go to download a 100GB file from some server and the download completes in 5 seconds, you'd be like "Wait, that.... just can't be right. I don't have a SSD, allocating the disk space alone should take longer than that..."

Of course, that just my assumption, and my impression.

And for what it's worth there are articles out there with more compelling alternative explanations, some of which were linked to from this article.

Try this one: https://mindhacks.com/2016/12/08/rational-judges-not-extrane...

This makes much more sense than your previous response.

When you have unexpected results, you need to examine you assumptions and the methodology behind them.

You can't simply dismiss the results out of hand. In your examples of code coverage, or fast file transfers, you need to dig deeper, not dismiss the results out of hand.

For instance, testing the Object.toString() function (although I'm not sure how you'd test that) is something that could result in abnormally high test coverage. Also, when you transfer a file quite quickly, is the OS doing any sort of compression on the data? Is it only for that file? How lossy is the transfer? Etc etc.

Simply dismissing the results out of hand because they disagree with your preconceived notions of reality is wrong.

For instance: https://medium.com/war-is-boring/eating-too-much-rice-almost...

Japanese scientists were convinced that the beriberi epidemic was due to germs rather than a nutritional deficiency because that was the prevailing notion of the day.

Based on this data, the difference between the height of 21-year-old men and women in The Netherlands is approximately 13 centimeters. That is a Cohen’s d of 2. That’s the effect size in the hungry judges study.

If hunger had an effect on our mental resources of this magnitude, our society would fall into minor chaos every day at 11:45 a.m. Or at the very least, our society would have organized itself around this incredibly strong effect of mental depletion. Just like manufacturers take size differences between men and women into account when producing items such as golf clubs or watches, we would stop teaching in the time before lunch, doctors would not schedule surgery, and driving before lunch would be illegal. If a psychological effect is this big, we don’t need to discover it and publish it in a scientific journal—you would already know it exists. Sort of how the “after lunch dip” is a strong and replicable finding that you can feel yourself (and that, as it happens, is directly in conflict with the finding that judges perform better immediately after lunch—surprisingly, the authors don’t discuss the after lunch dip).

It is impossible because we have very strong evidence against it.

The flaw is that the leap from decision in difficult cases to dangerous driving or other behavior are wild guesses. He sets up straw men versions of what the impact of the evidence should be, then rebuts them.

The reality is that the same psychological state that makes a judge less lenient might also make us better drivers, not worse.

You're missing the point. The point is that the effect sizes in question are large enough that people should notice them in their day to day lives, because the proposed mechanism of action should not only apply to judges, but should apply to everyone everywhere. But we don't see this big obvious affect in other places.

I'd like to postulate that it may be that such chaos actually exists now, it is just spread throughout the day as not everyone takes breakfast/elevensies/lunch/dinner at exactly the same time as their peers, and metabolisms vary from person to person.

> It is impossible because we have very strong evidence against it.

We do not have evidence against it.

Everything he has written is the _lack_ of things that he thinks should logically happen, rather than things that are happening that would disprove the hunger hypothesis.

Evidence would some other experiential data, such as the number of birdies in golf remaining the same before and after lunch.

>Everything he has written is the _lack_ of things that he thinks should logically happen, rather than things that are happening that would disprove the hunger hypothesis.

Actually arguing on the "lack of things that should logically happen" is exactly the same as arguing on "things that are happening".

In other words, what he says is the same whether he says "It can't be happen because X would be happen too and we don't see X", or he says "It can't happen because we see Y" (where Y is not X).

> Actually arguing on the "lack of things that should logically happen" is exactly the same as arguing on "things that are happening".

Reasoning from first principles is a huge downfall the higher up the science stack you go. The only time you really reason from first principles is in math. When you get to psychology and sociology, you need try fit observed data to hypotheses.

The author has suggested interesting follow up experiments, but has given no backing for the disproval of the hypothesis other than "I don't believe that can possibly be true."

That's not science, or even an informed opinion on science. That's just religion.

>Reasoning from first principles is a huge downfall the higher up the science stack you go. The only time you really reason from first principles is in math. When you get to psychology and sociology, you need try fit observed data to hypotheses.

Observed reality/experience -- which is what the author invokes -- is not "first principles" though.

> Observed reality/experience -- which is what the author invokes -- is not "first principles" though.

You're missing a huge point though, which is the difference between "observed reality" and calling out BS.

The author simply says that all these other things should happen if the hypothesis is incorrect, without any experimental data behind it. There is no evidence behind his assertion that all these other things should happen, so he's reasoning from his life experiences and his personal biases.

>The author simply says that all these other things should happen if the hypothesis is incorrect, without any experimental data behind it

That's still not first principles though -- not in the math/logic sense.

It's just things he doesn't bother to justify because they are well established.

A new paper in Physics doesn't begin by trying to prove the relativity theory or to give citations that show it's true: it just takes it for granted.

I completely agree. The article's author suggests that the study's result is too good (to be true), and casts that as the sole reason to reject the study. I think that such a surprising result definitely deserves scrutiny, but barring problems with the method or stats used, it's merely surprising, not "impossible".

Within the paper itself, the authors clearly considered this:

  A key aspect for interpreting the association
  between the ordinal position of a case and parole 
  decisions is whether an unobserved factor determines
  case order in such a way that yields the pattern of
  results we obtain.

They then interview judges and lawyers and examine the process to rule these out! Which is rare enough for a quantitative study like this (where they often publish the conclusions and just say "it could be X"). Unless the author can cite a more specific grievance, I'll trust the data.

> I think that such a surprising result definitely deserves scrutiny

But I think the point of the OP is that the effect is so large, it would be impossible for it to be a surprise to anyone.

The beginning of the article has several links giving reasons why the correlation could be explained by other effects. This article explains why the correlation can't be explained by the given reason.

> This article explains why the correlation can't be explained by the given reason.

The correlation exists, and the authors of the paper suggest a possible hypothesis that would explain it.

This article explains nothing - just goes into flights of fancy describing possible outcomes based on the author's logic. If there were actually studies that show leniency vs. hunger that contradict original findings, that would explain why the correlation should not be explained by the given reason.

In fact, the referenced study is a perfect example of a good science paper, and the scientific process in general. "Hey, we have a big anomaly here, here's a possible explanation. This explanation also suggests other further papers/studies that could explain the problem."

Well it's probably not a bad paper. And the follow-up from the authors seems pretty solid too. http://m.pnas.org/content/108/42/E834.full But that doesn't mean it's right. And yeah the linked article isn't going to get published in PNAS or anything but it's an interesting thought experiment.

Look at Bem's studies. They highlight this phenomenon clearly. Even meta studies can't prove things.

http://daniellakens.blogspot.com/2015/04/why-meta-analysis-o...