That may be representative of the wider society, but amongst my acquaintances it is not.
The methodology is outlined here:
If so then even worse. They'd only be working with people that take snail mail seriously.
If you have tax problems, the I.R.S. will not e-mail you. It will not call you. It will not text you. It will not contact you on Instagram, Facebook, Twitter, YouTube, Slack, WhatsApp, or any other digital method.
It will contact you via snail mail.
Serious people take snail mail seriously.
Really, though, I wish people who question survey methodology would mention specific problems with the methodology.
One problem I see is that part of the weighting is based on "ACORN" categories, which aren't well-defined and remind me of junk sold by marketing companies. Of course, I assume the data's based on actual demographics, so it might be good for weighting even if it's bad at explaining.
>At each address respondents for interview were selected by asking the person who answered the door if it would be possible to interview the person normally resident at that household ages 14 or over with the next birthday.
If birthdays weren't known by the door-answerer, then the interviewer picked a subject by first letter of the first name. But that only happened in 5% of cases.
I've read that section a few times now and I can't see even a hint of that implication. Is it because addresses are chosen randomly from the Postal Address File?
Not necessarily.
While it's common for people on HN to eschew the results of surveys because of presumed sample bias, that's rarely what happens.
I took a couple of statistics classes in college that spelled out the math involved, and essentially if you survey enough people within a population of a certain size, unintentional bias gets reduced to the point where it becomes a non-factor.
It's all very interesting if you're into math. I am not, but statistics courses were required for my journalism degree.
("Unintentional" meaning there is such a thing as "intentional" bias. For example, if you're deliberately surveying Toyota owners about Toyotas, or something like that.)
[1] https://www.ons.gov.uk/peoplepopulationandcommunity/populati...
When the United States had about 300,000,000 people the number for statistical veracity was around 975.
Again, my statistics classes were a long time ago, so the numbers I remember are inexact. But once I groked the math, it was surprising how few people you needed to interview.
In this particular case, everything you've said is true only if you assume a reasonably random sample. If your sampling method itself introduces biases, no increase in sample size is going to eliminate those biases.
I'm not enough of a statistician to feel particularly confident in passing judgment over the sampling method here, but I think it's fair to question it, despite the fact that I'm ordinarily far more likely to take the statistics at face value than the average internet commenter.