>With caller id and modern cell phones, I imagine there are pretty big demographic bias in who will willingly participate in a survey, and I suspect their outlook will dominate and may not be representative of the populace? Do they have a realistic way to collect reliable data in modern era?
All well-run surveys will apply weights to responses when calculating rates so the results reflect what would be if the participants were a "perfectly distributed" sampling across groups.
The article links to another article[0] for the consumer sentiment survey. The linked article says this:
>Methodology
>Bankrate commissioned YouGov Plc to conduct the survey. All figures, unless otherwise stated, are from YouGov Plc. Total sample size was 2,000 U.S. adults. The margin of error for total respondents is +/-2.43 percentage points. Fieldwork was undertaken between September 28 – October 6, 2023. The survey was carried out online and meets rigorous quality standards. It employed a non-probability-based sample using both quotas upfront during collection and then a weighting scheme on the back end designed and proven to provide nationally representative results.
2000 isn't a bad sample size for the basic analysis done. But I wish they'd include errors when comparing subgroups. Some of these differences are probably not statistically significant.
YouGov draws participants from a pool of people who've already been recruited and tracked[1]. I haven't found a good explanation of how they recruit or retain them, and don't feel like spending much more time on it.
There's one last red flag with the survey quoted in the Bank Rate, and it qualifies for my rule of thumb to ignore the article: the survey is not linked. I'd be happy with just the methodology, executive summary, and technical notes. Otherwise, I assume anyone quoting a survey they could but won't share isn't being totally honest.
[0]https://www.bankrate.com/personal-finance/biden-economy-and-...
[1]https://yougov.co.uk/about/panel