Hacker News Reader

Comment by mabbo | Hacker News Reader

11 adults in the experiment. No control group (other than against themselves). Experiment ran 8 days.

I mean, I'm interested, but I want to see further research before I call this a real result.

While it’s often an ostensibly appealing point of criticism, it’s important to note that sample size is just one aspect of study design—the investigators used a crossover design, which substantially increases statistical power. In the Methods, they state that 10 subjects are sufficient to achieve 80% power, which (criticisms of NHST aside) is a generally accepted standard, i.e., to obtain NIH funding for a study, you must demonstrate in your grant application that your design achieves that 80% power benchmark.

I always appreciate how the HN crowd evaluates the quality of the evidence, not just the quality of the headline. Great filtering function.

There's usually one good post. The rest are almost always people sharing their diet stories. Or if the headline is about exercise, you get reams of stronglifts or 5x5 or whatever, nobody cares. It's what engineers tend to do when they're presented with a topic they don't have actual expertise in.

Is this specific to engineers? I noticed this in college classes across disciplines. A lot of the time that someone raised their hand, it was to share an anecdote and not a question.

I find it honestly surprising that I see it so much on HN. You have a group of people who largely identify themselves as being rational, objective, intelligent people - datascientists, engineers, etc. - who stereotypically I would think to be naturally sceptical and want to examine evidence closely.

But it's absolutely true, the minute some scientific study is posted there are 50 people writing comments that they believe it because yesterday something similar happened to their aunt etc. It's something I've never quite figured out.

It's because HN users aren't really that intelligent; they just give off that impression instead. We're all mostly normal people here, and act like normal people. This isn't some special club of 1000x geniuses.

Even if this is true, each of us has a moral obligation to believe the very opposite re: ourselves.

Case studies can be very enlightening but there are also problems. Stories may be made up or misremembered. There is also a selection bias in which stories get told and which don't.

The big benefit I think is that people are... well people. If you do the same thing to an atom it will respond in the same way, while people may respond differently depending on any number of factors.

Appetite is known to be heavily entangled with psychology. People eat to celebrate. They eat to celebrate or because they are sad, because they are tired, because they are stressed out, because they are bored or maybe because they passed a bakery.

Case studies can help us understand how all these things interrelate and how to design studies that avoid pitfalls.

My favourite version of this is when a scientific study comes out and you get lots of posts from people saying, effectively, I disagree with science based on my feelings. The difference between anti-vaxxers or creationists and the average HN poster isn't as big as many pretend it is, I think.

The problem with this is that in many situations, you can have a perfectly valid study done with only 8 people. It entirely depends on who those 8 people are.

You are by far much more at risk of falsely believing something because your heuristic for believability is wrongly associated with skepticism.

Could you elaborate on the no control other than against themselves aspect?

I read this in the original paper, “Participants were randomly assigned to follow either the control schedule (eating between 8:00 am and 8:00 pm; 12‐hour daily eating period)”

Other than the N is on the lower side, how’s this way of setting up a control group insufficient?

The control group needs to be thinking they're doing something that works. If it's obvious they're in the control group, then it's no longer a control group.

It’s still a control, just not blinded to the participants (which, yes, can result in bias).

Yeah for sanity. Sad I'm having to post this and vote.

Title says "appear to...", not "does...", so there's that. Semantics counts.

Reply on news.ycombinator.com