Hacker News Reader

Comment by pmoriarty | Hacker News Reader

"Data was collected in 1 day. Is this a joke? No it's not!"

What makes you think data collected over 1 day worse than the same data collected in multiple days?

"I really don't understand how they can do a half-assed data survey on 258 people only, and then have the gall to claim "83% of DEVELOPERS""

It's called extrapolation.

No study ever interviews everyone. They all have to extrapolate from a subset of the entire population.

Obviously the more participants there are the more confidence you can have in the conclusion, but whether 200, 2000, or 20000 participants would make for a "good" study is completely arbitrary.

View on HN

> They all have to extrapolate from a subset of the entire population.

This only really works if you have a representative subset of he population you're extrapolating to. Is the distribution of experience roughly equivalent to the general population of "developers"? Age distribution? Company size? # of children? Home life? These kinds of demographics can have a big impact, and I see no discussion of how they were accounted for.

Heck, even the general work ethic culture difference between the UK vs the rest of the world could be a huge confounding factor that I know wasn't really accounted for.

"This only really works if you have a representative subset of he population"

Of course. Never said otherwise.

In the context of your comment it is implied that you believe the "study" in question does have a representative subset.

while you may not intend that, most people reading your comment would believe you think the study is fine

"In the context of your comment it is implied that you believe the "study" in question does have a representative subset."

No. I was replying to the specific objections the parent poster had.

I never said or implied anything about whether I thought it was a good study or not.

Clearly, several people (including me) think that you did imply it - so, regardless of whether or not you intended to do so, that's what you communicated in your post.

In particular, the way in which you mentioned extrapolation in the context of the study (and the conversation in general) carried the implication that the extrapolation was valid.

"Clearly, several people (including me) think that you did imply it"

Wow. Several people on the internet think something.

Is that a large enough sample size?

Getting a perfectly representative subset of the population, means surveying the whole population of the world. Also, accounting for every imaginable variable is always impossible, because we cannot ethically grow people in an isolated environment.

Sure, a meta-analysis of several studies reveals more than one single study, but that doesn't make a single study bad or worthless - or as said in the parent - "a bullshit study".

There are plenty of ways to account for not having a representative sample distribution, I'm just saying that I didn't see any kind of discussion about demographics or confounding variables in the actual study.

This was a pilot study, a way to open an avenue of discussion, and to be sure I think the study warrants more investigation. They even say so: "In future we would like to conduct larger studies over multiple countries."

Nit, the 258 number makes this an inference on the broader population, not an extrapolation, if it's a random sample.

It is, however, an extrapolation to make claims about the entire dev world when looking just at UK devs.

You always want at least a week of data collection, to control for weekly effects. People are often happier on a Friday than a Monday, for instance.

I'm not sure how much of a difference the day of the week would make for people who are really burnt out.

True burnout, probably not, but it's always better to avoid problems like this by collecting data for at least a week.

it wouldn't, it would make a difference for the people who aren't burnt out

The question is precisely whether they are "really" burnt out or just had a long day (or couple of days) at work and are feeling burnt out at that particular moment. We all experience something akin to burnout at some point or another but it makes a big difference whether it is "I felt burnt out for a day or two after a tough sprint or major production incident" or "I've been working 80h per week for months". The former you might be fine after a relaxing weekend whereas the latter may take months to recover and seriously jeopardize your long-term physical and mental health.

Another study finds out that 83% of population are old ladies with a cat that happily respond to surveys by post.

Reply on news.ycombinator.com