About those baby brainwaves
cspicenter.org
cspicenter.org
Having said that, using a study done in 2022 (which we're only just out of the first quarter of) to justify basically anything, seems kind of ridiculous, and I'd like to live in a world where people just laughed that off in the same way they would if someone said an astrologer predicted their policy would be successful or if Joe Rogan was offering medical advice.
And that's before we get to the brainwave part, which seems to push this into the ridiculous.
If we really have no better research to suggest how to improve society in general or child welfare in particular, then that alone seems like a sad state of affairs. We have a whole globe full of children, more than 2 Billion of them in various different locations and societies, have we really not got any way to identify what is helpful. Are we claiming nothing at all that happens to any of those 2 Billion children makes any difference? That seems very unlikely. Or that we already do everything we know to do? This feels only marginally more plausible.
In this case, and most others, yours is the correct reaction. But sometimes you don't have a choice but to be hasty, as with rapidly-developing situations (e.g. COVID).
> Thus, we ought to be skeptical of social scientists’ ability to reliably inform public policy, and of policymakers’ ability to evaluate social-scientific research objectively, particularly when it is “policy relevant.”
Think of how much we’re debating what’s causing the current inflation we’re seeing. That’s one number in a field that’s quite amenable to quantitative analysis. I think we have pretty much zero credible scientific evidence to justify most policies involving public welfare. There’s a handful of exceptions—e.g. the tremendous work done studying the benefits of the clean air act, for example. But that was rigorous research done by the EPA over literally decades.
I think we should give poor families with kids money because that seems like the morally correct thing to do. But this sort of thing is just scientism: https://calteches.library.caltech.edu/51/2/CargoCult.htm
https://statmodeling.stat.columbia.edu/2022/04/02/why-the-ne...
He also suggests that he broadly agrees with the post he links to and that it gets a lot of ideologically motivated political pushback so this seems like the Ying to his Yang, an example of politically motivated disregard for solid science just going in the opposite direction.
Not to mention this paper notes in its conclusion that the study in question was already roundly criticized at the time (see the paragraph beginning with “[t]he public arena is at least somewhat self-correcting”), so it’s not clear what is being added to the discourse, other than unsupported generalizations.
Submitted title was '“Policy-Relevant” Social Science Is Mostly a Fraud', which wasn't so bad - it's indeed what the article title says - but maybe it's misleading on the article's own part.
(I expect it's part of a larger pattern, but the article doesn't really show this.)
Here's for example a second most recent post: The journal Public Choice goes the way of Lancet and publishes a paper that is “riddled with errors” but comes to a political conclusion that they want to support. https://statmodeling.stat.columbia.edu/2022/04/03/the-journa...
Replication crisis in social science is a very real thing, and it's not a problem with a few odd studies: at this point, at least double digit percentage, if not outright majority of studies in social sciences, are simply untrue.
Here's, for example, a quiz https://80000hours.org/psychology-replication-quiz/, based on a study published in Nature ( https://www.nature.com/collections/nfkchhxllx ), which tried to replicate some 21 experimental studies published in Nature or Science in a few previous years. These were not some crappy papers published in pay-to-publish journals: Nature and Science are the top journals, as high quality as you can get. Still, only barely over a half of the studies replicated! In less renowned journals, the ratio must be even worse.
What’s surprising is the impact on policy low quality have. Government policies Kees and the news, like KQED and others routinely refer to these kinds of studies when they espouse something and politicians and some of the twitterverse et all latch on to these things have very little validity if any.
Whether that be because they want to get tenure, enact change in the world, build prestige for their company to improve hiring/stock prices, or get a book deal.
1) Solving concrete problems (desalination, energy storage, flying like a bird, getting to the moon, etc)
2) Discovery for its own sake, sense of power over nature
Both of these can be pursued safely without a toxic CoI.
For example, if you need money to eat. Or, if you want to solve a concrete problem/discover things, and believe that if you got funding you could do so. Or if getting tenure would mean you get to solve more concrete problems, and fudging one paper would let you do that.
It's not possible to avoid conflicts of interest categorically. It is possible to manage the impact of them. For example, the peer review process makes it harder (but not impossible) for a lone researcher to publish inaccurate findings.
The question isn't whether there's a conflict of interest in the research, the question is whether the system publishes solid research despite the conflicts of interest.
Here is the real tragedy:
None of those outlets will report on this analysis and their proponents will beat us over the heads with "Trust the Science!"
The one I've found that works well is ignoring the deceitful liars.
That doesn't work if a large number of your fellow citizens believe them.
Even policy relevant social science research can produce good studies in highly contested fields, because of the mechanisms of peer review and open debate.
1 - http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.147...
The reproducibility crisis is more related to social psychologists, which include lots of nuts. They are not what I was talking about.
You are making a non-objective argument in bad faith to grind your axe.
The issue instead is that conclusively studying human behavior is hard, and effect size is often small, and it's inherently impossible to control for conflating factors.
Cult ideologies have no inherent properties that would cause the same, and yet no sane person would expect cult members not to shill for your their cult. Corporate shilling doesn't prove the opposite. It proves how easy it is to game soft science.
Personally, when someone makes reference to "policy relevant social science," I always ask myself: if the study concluded the opposite, would this person's opinion be different? The answer I come to is almost always no.
This[1] is an example of it which borders on Swiftian satire... [1] https://cspicenter.org/reports/the-accuracy-of-stereotypes-d...
>----------------------------------------
2. The Great Awokening
In the last decade, white liberals in America have shifted far to the left on issues of race, gender, and sexuality. Ideas once seen as radical, like defunding the police and gender being on a spectrum, are embraced by more people than ever before.
Social scientists know little about the social and psychological factors underlying this “Great Awokening.” Is wokeness a status competition among elites? An extension of WEIRD (Western, educated, industrial, rich, democratic) morality? Or something new altogether?
CSPI is interested in funding scholars studying woke attitudes, beliefs, and behaviors. We are also interested in work examining the impact of woke policies and practices on organizational culture and performance.
>-------------------------------------------------------------
It seems trivially true that supporting parents especially at the low end of the spectrum provides them with less stress, less home insecurity, less food insecurity, more likelihood that one or both parents can spend less time at work and ergo more time with the child.
If the actual question is does giving money to parents improve childhood trajectories the answer appears to be much more likely to be yes. Attacking a study that tries to go about proving this in a probably wrongheaded and silly way appears to be attacking sidelong a strong opponent via a weak front even if the stated aim is to pick on bad studies that indeed that studies peers already debunked as if it was representative of the entire field.
Here is a fun exercise do a search for site:https://cspicenter.org covid some fun results
https://cspicenter.org/blog/waronscience/why-covid-19-is-her...
Almost a year ago they said SARS-CoV-2 is going to become mostly harmless
Then they used Sweden to prove that lock downs were pointless ignoring how much worse they did per capita than Norway and Finland.
It's like fox news went to school and got an education.