Hacker News Reader

Comment by PerfectElement | Hacker News Reader

I don't see how this can be a valid scientific experiment if the control group is aware of their colleagues working 2 hours less and getting the same salary. How can this not affect the results?

View on HN

I think probably the only way to do it correctly is to divide the participants into two groups. In group A, they work 30 hour weeks for 3 months, then work 40 hour weeks for 3 months. In group B, they start with 40 hour weeks, then go to 30 hour weeks.

You can't actually hide control/experiment status from the test subjects, though. You usually know if you are working more or fewer hours in a week. There is some danger that people will intentionally slack off while doing 40s and bust their ass while doing 30s, just so the experimental results suggest shorter work weeks. To counter that, you have a huge number of people following the protocol, and the researchers and participants simply don't know exactly whose data will be randomly selected for inclusion in the study.

I am much more concerned about the volatility of small sample size. Sickdays inside a single building can be easy effected by single individuals. Imagine if either the control or test work place has an employe who is carrier for the flue but with limited symptoms. Such event would invalidate the experiment if one is simply looking at the number of sick days for data points.

At best, I think one should view this experiment as a preliminary test in order figure out how to do a real test with a larger sample size.

Not just small sample size, it's also biased. FTA, "The year-long experiment will compare two factions of municipal workers." Obviously, the results would be very different for various professions. For people who just push papers, yes they can be 25% faster to get 2 hours more with their family. However, what happens with the service industry? If the bus drivers work 2 hours less, then there will surely be less busses on the road.

Given the postscript, I'm not sure how upset the control group will be:

> A similar experiment involving 250 workers in the Swedish town of Kiruna was scrapped in 2005 after 16 years. With shrinking hours, job pressures intensified, reports The Local, and as a result, the city council concluded that sickness actually increased.

I feel like this is the case for most economics studies I've read (note: not nearly an expert). I enjoy the types of studies done and think they have value, but am concerned they can be taken as direct, strong supporting evidence when by their nature they make controlling for confounding variables impossible.

Double-blind is neither necessary nor sufficient for a "valid" scientific experiment. It just helps keep the variables down.

Sure, but this whiffs of cargo cult science. Given that the working conditions, workload and motivation of the "control group" are all heavily affected by the length of the work day of their colleagues in the other group, the interaction effect actually multiplies the variables and the experimenters would actually be more likely to make reliable inferences without it (i.e. measuring 8 hour workday productivity and health indicators as a baseline prior to introducing the 6 hour work day)

The main issue with that is, that the variable introduced by not doing an experiment double-blind can only be measured by repeating the experiment double-blind.

Reply on news.ycombinator.com