Comment by timr | Hacker News Reader

Comment by timr | Hacker News Reader

Parent Full thread

That paper is a joke. Here's how they collected their "treatment set":

> We collect RTO information for S&P 500 firms through manual news searches on Google and Factiva. Specifically, for each firm, we search by using the company’s name and the following keywords: ‘Work from Home’, ‘return to office’, and ‘days required working in the office’. We focus on S&P 500 firms because these large firms are closely followed by media, thus mitigating concerns about media coverage bias. Among the S&P 500 firms, 137 publicly announce their RTO policy and are classified as RTO firms in this study. These 137 firms become our treatment firms

They then correlate this with other (ahem) high quality datasets...like Glassdoor reviews.

Setting aside all personal biases, this is like bad undergrad research. You can't just look at titles and treat papers like pokemon cards.

You can tell it's a joke from the second sentence of the abstract. They don't even attempt to pretend that they're impartial or scientific, it's 100% obvious they're coming into it from a completely biased position.

"this is like bad undergrad research."

Unfortunately I think you just described the corpus of behavioral science over the past two decades

Sadly, yes. I mainly looked out of a sense of morbid curiosity, and I was not disappointed.

What is your concern with this approach? It seems pretty reasonable to me. Obviously the better the data the less chance for some kinds of errors, but nothing here sticks out as particularly bad. (Full disclosure I didn't read the paper itself.)

Among many other things, sample bias.

Sample bias in the sense that they may miss companies in the S&P 500 that aren't as well reported on (possible but I would guess not super likely), or that fortune 500 companies are not representative of companies overall (certainly true but still a reasonable scope for analysis)? Or something other than that?

People generally leave reviews because they are unhappy with a thing. That alone is sampling bias from Glassdoor. Rarely do people leave a review because they are happy with something.

Why is that a bias? When people are more unhappy, they post more negative comments. If they are less unhappy, then they post less negative comments. The bigger concern is that employers post fake positive reviews, which biases against the findings.

I know researchers especially junior researchers are very critical of others’ work. But they focus on s&p 500 because theses firms are big and closely followed by media.

Reply on news.ycombinator.com