It feels like epistemic weaksauce to claim that entire fields explicitly reject the goal of generalizable knowledge because they question or reject “the scientific method” on the basis of “I’ve read some case study / qualitative papers”.
Psych is getting a brunt of this because they have actually done a good thing and funded replication studies, which naturally will produce a bunch of "this fails to replicate" findings. So then you get headlines talking about the replication crisis in psych and then people who maybe took a single class in college a decade ago dismiss the entire field as bogus.
I invite such people to go speak with some CS academics for a better understanding of the mess in our own field.
I don't reject non-scientific scholarship. But I see the focus in some disciplines a little like twinkies and soda: a bit of them can be fine and maybe even be good in some circumstances (brings joy; maybe as a recovery item for some diabetic conditions). But my feeling is that some of these disciplines have indexed a bit too much on twinkie and soda.
I think of science in much the same way that you think of other disciplines.
1. I definitely did not say that economics does not strive for generalizability. Quite the contrary. I think the vast majority of economists strive for generalizable conclusions (i.e. go to great care to find study samples that are representative of the population of interest and use statistical methods that might allow them to plausibly conclude things about that group)
2. I never said "all" work in other social science disciplines rejects generalizability as an aim. However, I do believe, that more "empirical" scientific method based studies in a number of disciplines would be good. That is predicated on 1. a belief about the prevalence of qualitative/theory based scholarship. 2. a normative preference that the ratio is undesirable. I could be mistaken on my perception of the first point, but I don't think I am. On the second point, you're welcome to disagree. These are, like, opinions, man!
3. "reject the goal of generalizable knowledge because they question or reject 'the scientific method'" makes me think you've either missed my point, or you don't understand the nature of other forms of academic scholarship.
I'm not leveling a diss that e.g. a participant observation case study isn't a method intended to generalize from. That's just an intrinsic feature of that kind of study method. Of the qualitative researchers I've known, I can't imagine any thinking there is anything controversial about what I've said on that point. Though, some would definitely disagree with me on my opinions about what ratio of scholarship should be of this kind.
And lastly you've selectively quoted me there at the end, and attached a conclusion to it that is not mine. Consider this a bit of original qualitative research on my part: a sampling of non-empirical research that I've read in recent years has suggested to me a lapse in rigor in certain disciplines. But this conclusion is grounded in my "situated knowledge" of the space, and thus shouldn't be used to generalize without a suitably operationalized quantitative study ;-)
Pragmatically speaking, I'm unsure why you would choose this language if you wished to convey the nuance that qualitative methods are actually great, that you're merely wishing that more "empirical" studies would also be undertaken, where "empirical" I guess means "quantitative" and "rigorous" though you don't make that explicit.
For what it's worth, you absolutely can generalize from an observational case study. RCTs are not the only way of drawing generalizable conclusions—it depends a lot on what your epistemic goals are.
It kind of sounds like you don't like that some social sciences rely more on non-quantitative methods because you don't think those are definitive. That's fine, you're welcome to hold that belief, but let's not pretend like you're a fan of all methods and just wish there were a few more quantitative studies in sociology (or whichever discipline).
Does that resolve what I think you are perceiving to be a contradiction, but that I do not see as a contradiction?
Where I think you and I have a fundamental disagreement is in the nature of e.g. qualitative research methods such as case studies and whether a case study is generalizable. This feature was taught to me by... qualitative researchers. If you truly believe a case study is a generalizable research method, then either you are defining generalizability in a different way than is typical, or you hold a minority viewpoint not shared by mainline qualitative researchers.
Not all research methods need to be generalizable to have some scholarship value, and I'm not using the concept of generalizability as a colloquial stand-in for ~"bad"
BUT, I still think social sciences could use fewer qualitative studies and pure theory grounded scholarship (i.e. "a correction is warranted"). It's what I said from the start, it's what I mean now, and unless I someone provides me a compelling rationale to change my mind or the world changes, it'll probably be what I believe going forward.
It seems important to you to attribute to me a total rejection / disparagement of these methods. I am just not saying this. I do not reject these methods, just as I do not reject apple pie. I just don't think that a steady diet of apple pie is good. And I don't think the research methodology diet of many social sciences is healthy either. A correction is warranted.
What will be the outcome of that shift? Some kind of better research?
On what basis do you think that field should agree with your perception?
On what basis do you think your perception is correct?
Note that I’m using “qualitative” and “quantitative” as stand-ins for whatever you think there’s too much of and too little of, respectively—please feel free to clarify if these words don’t effectively capture what you are trying to say.
It’s not at all important to me to attribute to you a “total rejection” of these methods, your writing implies to me that you have a preference for quant methods and think they’re better. You’re not, for example, complaining about too much quant methodology in economics, and too little qualitative. Sure, you don’t think that qual methods are bad per se, but you do think that the social sciences could use fewer of them, and that they’re overused.
I imagine you would agree that these methods can tell us different things, and that they’re not interchangeable for any given research question. You’d probably also agree that some fields bias towards certain types of questions, and that maybe the ratio of methods of work in a given field reflects the bias towards questions that are best answered by those methods. So, are you suggesting that entire fields should focus more on different questions, specifically those that can be answered quantitatively?
If not, I’m not sure I understand what the implications of your argument are.
For what it’s worth, here’s my bias: I am both a quantitative and a qualitative researcher, and I actually think the underlying issues holding back the production of generalizable knowledge have little to do with choice of methodology, and to the extent that they do, it’s in part due to a fetishization of quantitative methods that tell us something generalizable—but not necessarily something useful or even something true.