Hacker News Reader

Comment by mattw2121 | Hacker News Reader

"For this campaign, we surveyed 930 Americans to explore their retirement plans. Among them, 16% were retired, 22% were still working, and 62% were retirees who had returned to work."

So, 149 of those surveyed were retired. Of those 149, 25 (1 in 6) are considering returning to work. 13 of those want remote positions.

View on HN

Thank you for extracting the actual numbers. Constructing underpowered surveys with loaded questions is a common tactic for pushing narratives like this. The headline confirms a lot of people’s beliefs that economic conditions are forcing people out of retirement and that remote work is the way of the future, but it really just a survey of a couple dozen people who are “considering” and “wanting” something, which ultimately has little bearing on what people actually choose to do.

These underpowered survey tactics are most frequently used in “__% of Americans are living paycheck to paycheck” headlines. That’s why all of those headlines have drastically different numbers, some as high as 70-80% (which is obviously untrue). It really just means that someone was able to get a few random people on the phone to answer some questions in the middle of the day, which isn’t representative of the population at all.

I think the standard error for this survey is only like +/- 3%, which imo doesn't change the narrative THAT much. That said, without more details on the survey design, it's hard not to think issues like sampling and response bias make the results difficult to generalize...

I work in quantitative research. A cell with 13 or 25 respondents in it ought to be used with caution, but is not necessarily underpowered. My "rule of thumb" is that, given a well-represented and properly weighted sample, having 30+ in a given cell when crosstabbing like this is healthy, and 10+ per cell is the bare minimum I will accept - with caution.

Well-represented and properly weighted samples are unusual. As often as it’s used, phone surveys don’t qualify.

why is it obvious that that is untrue?

On this site, the null hypothesis is whatever we already believe. When we see evidence of an alternative hypothesis, we find ways to throw that evidence in the trash. Small sample sizes, replication crisis, misleading headlines, research on mice, etc. are all ways to ignore whatever evidence we want.

In short, we are always right, research be damned.

There is evidence that people are pretty good at predicting whether or not a study is going to replicate, so they're not necessarily off Ba(ye)se:

https://www.theatlantic.com/science/archive/2018/08/scientis... https://www.newscientist.com/article/2218843-common-sense-ca...

(in this particular case, I think the fact that this is content marketing is what is causing people to react so skeptically)

This pretty much highlights what I'm saying. Regardless of how strong or weak this research is, it won't be scrutinized by viewers here, because it confirms what they want to believe.

In the parlance of my country, if someone is "living paycheck to paycheck" they're a member of the working poor, people with no savings, desperately fighting to keep their head above water.

Here is a survey saying 36% of Americans making >$200,000 are living paycheck to paycheck: https://www.lendingclub.com/asset/ls/cms/media-center/12th-p...

There might be a handful of people earning that much who are in a difficult financial position - divorced airline pilots with expensive alimony payments, perhaps? - but it is extremely hard to believe that 36% of people earning 4x the US median income are the working poor.

On the other hand, the 36% number would make much more sense if the survey had asked people "Do you depend on your salary to pay for your living expenses?" or "Do you have enough savings to pay for 3-6 months of basic living costs?" i.e. much more expansive definitions of what it means to live paycheck-to-paycheck.

> it is extremely hard to believe that 36% of people earning 4x the US median income are the working poor.

I don't find that hard to believe at all, at least in the US. That income bracket tends overextend themselves and carry too much debt. A shockingly high percentage of people in it will be bankrupt in 5 years.

No, you just don't understand how statistics works. If you get 80% on sample of 100 people, your 95% confidence interval is +/- 8%. This assumes a random sampling method.

> This assumes a random sampling method.

I would encourage you to explore how difficult it is to actually get 100 responses from 100 randomly selected Americans, and how non-response/self-selection affect your confidence interval.

It is difficult. That is irrelevant to the bad logic I was responding to.

I think this discounts the finding that 575 people, the majority, were retirees that already returned to the workforce. This 2011 study said the rate for this was 15%. https://www.bls.gov/opub/mlr/2011/article/reentering-the-lab...

That's not a "finding". It was their choice to survey mostly retired people who had returned to work.

IMO it's less about power than the self-selection of the responders. Naive MoE estimates that assume random sampling completely miss that respondents are just a difference species, esp. w.r.t. financial attributes.

It would be shocking if some large fraction of high-earners really were living paycheck-to-paycheck. That a large fraction people who chose to respond to a survey and who stated they earn $xxxK/yr also said they're living paycheck-to-paycheck is ... somewhat less surprising.

There should be half a dozen "among survey respondents" comments to headlines like this, the same way there are a dozen "in mice" comments to every breathless report of medical progress in this-or-that.

Are we using "considering" with a stronger definition? Otherwise this is meaningless.

On any given week I consider changing job, quitting, early retiring, changing teams, changing projects, etc... My brain will just rapid fire different scenarios and I'll typically conclude that staying put is my optimal move.

What's more surprising to me is that 5 in 6 are NOT considering alternate scenarios.

I "consider" running away to join the circus, at least, once a day.

"Software engineers on Hacker News are discussing joining the circus after mass layoffs" - paychex, probably

That's the research proposal I want a grant for. My work will be going to the circus all the time and asking the acrobats if any of them used to work for Adobe.

...And we don't know their respective jobs.

The only indicator we get is that more than 50% of those who consider going back need money, which probably correlates negatively with jobs that have a low entry barrier for remote work.

Add to this that people above 50 tend to have serious problems getting jobs in tech (and likely adjecent industries).

Is your point that the sample size is too small? Beyond a certain threshold, sample size doesn’t matter as much as you think.

That's true if the sample is random.

25 out of almost 1000 doesn't seem like much to me. My parents are both retired and financially comfortable but would consider employment with a lot of flexibility. The 62% who had already returned to work IS surprising to me. I'd bet most of those are financially motivated

Yeah. I imagine there are (broadly speaking) two groups that are conflated. There's the group that has returned or is considering returning to full-time work because they found they need the money. And there's a group who might do something interesting on the side for a bit of money and because it's interesting.

I would absolutely consider the latter in retirement but would hope not to have to do the former.

There should be a law where you have to state this sample size stuff at the start of every statistical article.

I wonder why they reported 53% interested in remote (rather than 52%, which is 13/25).

Reply on news.ycombinator.com