Why psychology isn’t science (2012)
latimes.com
latimes.com
> How does one measure happiness? Psychologists can’t use a ruler or a microscope, so they invent an arbitrary scale.
All scales are arbitrary. We measure length in meters for no reason other than the meter is a nicely human-sized measurement. I'd maybe even argue that the definition of a scale is that it's arbitrary, a mapping of /something/ onto the number line chosen in a way that the numbers are within "human" range.
The real difference being highlighted isn't anything to do with measurement, it's just a difference in dimensions. It's very obvious how we can compare 2 things in space-time dimensions, either they happened at different times that we can map onto our time scale, or they occupy different sizes in space that we can map onto our space scale. Emotions aren't quantifiable in these dimensions, but that doesn't mean we cannot create a space in which they are.
If the author would argue that happiness isn't a "thing" in the way "this piece of paper" or "1945" is, I think that argument can be had, and has been taking place over thousands of years of philosophy. To not even raise the question, but (in my eyes) to assume the answer as obvious, is, as I said, lazy writing.
> If I ask you 3 seconds apart, though [...] would you not expect consistent results?
I completely agree with your original comments about lazy writing by the OP. But these all seem besides the point.
Our bodies and minds go through all sorts of states, some cognitively introspectable, some not.
"Happiness", whatever it may be, is what it is and need not conform to your presumptions. That's the point of studying it. Nothing about its fluctuations or the way these are felt by an individual are a problem per se with the research. It just means it's hard to do, and we may not know a whole bunch about it.
As far as I am aware, there is no psychological measure/test which is not subjective. Please correct me if I'm wrong.
To me, this is why neurology is hard science and psychology is soft science.
This point seems to slip through these types of discussions, because people want to keep changing the focus of the research. When you change the topic from subjective experience to, say, neural properties, you're not studying the same thing anymore.
There are plenty of measures of subjective experience of happiness. We can debate the merits of these measures, but they exist. They're reproducible and you can scale them over time or across people.
If someone says they are miserable, and rates themself a 1 on a scale of 1 to 5, this has to be subjective. Are they lying? Perhaps, but then too maybe that radio signal you obtained is a microwave next door.
Units on many psychological measures could be obtained (see test equating) but there's not enough agreement on which measures to use, and in fact, use of multiple measures is encouraged because of variation in what might be an appropriate target of study (hedonic versus eudaimonic well-being for instance). Units in psychology are somewhat similar to where units of temperature were in the 18th or maybe 19th century.
Can you even do this with happiness? What is a unit of happiness?
(If you’re asking about the units for wealth: USD, Euros, etc)
So it's not like it can assess this number for an individual, at least not without first comparing and contrasting it with every other person in some sample group.
In other words, a true unit of happiness, would allow a completely independent measure of any one individual's current state of happiness. Imagine something like a thermometer for happiness, you point it at someone and it reads out the units of happiness (over-simplifying the application to make the point). Therefore we're not even close to knowing what this is or how to measure it, and the way we really communicate around it is by using our own subjectivity of it and making correlations.
The ladder question (a common tool for measuring SWB) is this:
Please imagine a ladder with steps numbered from zero at the bottom to 10 at the top.
The top of the ladder represents the best possible life for you and the bottom of the ladder represents the worst possible life for you.
On which step of the ladder would you say you personally feel you stand at this time? (ladder-present)
That being said, I’d argue “2001 dollars” is a unit, not “dollars”. You’ll often see data in econ journals cite something in that type of unit.
Another angle on this is that comparing wealth across time is nontrivial (and the money isn’t to blame). How do you compare the value of a iPhone in 2020 to a landline in 1980? It’s possible, but nontrivial. Our units (in some messy imperfect way) reflect that (and other factors).
Why though? Why dollars in a particular year, rather than dollars in general? Sure the value of a dollar changes from year to year, but the value of a dollar also changes constantly in any given year too. Over the past year, the value of a US dollar relative to a Euro has fluctuated by more than 10 cents. In fact the relative values of currencies fluctuate constantly, near-enough to instantaneously. But this isn't reflected in the pricing of most consumer goods; the price of common household goods in a grocery store might vary a few times a week, but not thousands of times an hour. So in one context the value of a dollar seems to vary constantly, but in another context the value of a dollar is substantially more stable. So a dollar doesn't even have a set particular value at any instant in time; it depends on what kind of store you're standing in.
If happiness is a metric with units even worse than this, then I see little hope for any scientific study of happiness.
Say what you want about stellar kinematics, but measuring meters simply is not so fraught as this.
For length, MSA can be easily conducted. For happiness, what metrology system are you going to use? What's the proposed measurement method?
[1] MSA: https://www.metrology-journal.org/articles/ijmqe/pdf/2010/02...
I go for a run most days and my watch asks me how I feel afterward. I don't answer, because I really have no idea whether my "Good" answer one day corresponds to a "Good" answer on another day. Often, it probably doesn't. In my view, talking about subjective well-being is anything but simple.
It’s subjective because it cannot be universally and independently measured.
It’s not possible to measure the happiness of two individuals such that I can compare them against each other quantitatively and define who of the two is happier.
I can do that, however, with basically every physical measurement.
Quantitative psych people are well versed in how to think about and model latent variables. They don’t go “oh shit! can’t measure happiness, I guess we’d better just make stuff up.”
I’m plenty critical of the research coming out of psych, but to pretend that the field remains naive to even the most basic principles of modeling is a bit ridiculous (and like you said, lazy)
So yes I do think the majority of psych researchers are making stuff up (through p-hacking and other nefarious means) in order to get funding and tenure
1. https://replicationindex.com/2018/11/20/how-replicable-is-ps...
Not to belabor this but to note the irony here: there is probably a similar psychological (ehem) phenomenon that happens to a psyc grad student who lets their presumptions bleed into their interpretation of a measurement.
> The first principle is that you must not fool yourself — and you are the easiest person to fool.
R. Feynman
That doesn't seem particularly terrible in comparison to scientific reproducibility in other fields. What am I missing? https://en.wikipedia.org/wiki/Replication_crisis
I don't know if arguments about "real" science are especially useful. The nature of science has changed a lot of over time and will presumably keep changing. It takes us into questions of whether fields that can't do experiments are "real" science, for example. It also assumes replicability is the only way science can't be "real" but there are some fields (e.g. COVID research!) where papers are usually replicable but that is meaningless because the methodology is unsound in other ways, i.e. you'd just be replicating their mistakes.
The biggest problems driving non-replicable research are the fact that so much "science" is now really just data analysis, and finding some spurious correlation that just about nudges over some arbitrary and very weak threshold for significance can be considered "science", even without anyone doing any kind of double check or followup. No real world experiment needs to be done at any point. And in some fields the connection to reality has been completely lost - public health is filled with papers that make predictions about the future trends in public data, and nobody ever follows up with papers that compute the correlation between the predictions and measured values. It would be easy, but they generally don't do it.
While you need „second“ to define the „meter“, the former is not derived from other units but from an observation of nuclear reactions in Caesium.
https://en.wikipedia.org/wiki/2019_redefinition_of_the_SI_ba...
It's sometimes poorly done, but this is true of all the sciences; malpractice and poor rigour in particular experiments don't invalidate an entire field. It's also sometimes misused by other fields, but this is (again) true of all the sciences - a therapist misapplying a principle doesn't stop psychology being a science any more than an engineer miscalculating makes physics pseudoscience.
Science is about the approach, not the topic or the perception.
Agreed on the approach part; as for the topics: there are topics which are inherently hard to "prove"/falsify, either by lack of sufficient evidence (existence of god(s), UFO-as-aliens etc.) or lack of tooling required to tackle the complexity of interactions (many large scale societal problems, psychology, (geo-)politics etc.).
And while "proper" study of those topics is definitely considered science, insisting that we soon can achieve breakthroughs is most likely misguided. What I mean is that we cannot even map and (even if crudely) simulate a single brain, let alone detailed-enough interactions between 7 billions of people. And that's what's probably required to get some solid answers here.
“Physics” isn’t, though, just carving out of particle physics as a distincg niche within it. But mostly the difference is particle physics deals with simpler systems in which the parts are easier to isolate.
> There is a qualitative difference between the sciences which generate predictive models and those which cannot.
No, there isn’t; all sciences, including psychology, generate predictive models. There’s a quantitative difference in the degree of uncertaintity expected in predictive models in various sciences which affects testing models, and is a product of the degree of difficulty of isolating the effect modelled from potentially confounding influences.
And hey, as someone who was a psych major I can say that psychology is a much more difficult science than either my wife's work or me trying to improve conversion rates, but that's just because it's dealing with things that are harder to measure than changes in conversion rates or the bindings of antibodies to targets. As long as you're correctly applying the scientific method to it, you're doing science.
Topics like this are, as a whole, not true. While I think psychology is no where near as problematic, the author’s argument that a loose definition of science, like whether the scientific method is employed, can be dangerous.
> This is dangerous because, under such a loose definition, anything can qualify as science. And when anything qualifies as science, science can no longer claim to have a unique grasp on secular truth.
Ultimately, what we define as scientific will impact what we perceive to be truthful. Setting a higher bar for a topic to be labeled scientific will strengthen our grasp on secular truth.
> But to claim it is “science” is inaccurate. Actually, it’s worse than that. It’s an attempt to redefine science. Science, redefined, is no longer the empirical analysis of the natural world; instead, it is any topic that sprinkles a few numbers around. This is dangerous because, under such a loose definition, anything can qualify as science. And when anything qualifies as science, science can no longer claim to have a unique grasp on secular truth.
On the subject of categories, they cease to be useful when the contents of the categories can no longer tell you anything about the category itself. The further you get down this axis, the less useful the category is. So any dilution of the strength of a category by overly broad inclusion should be scrutinized.
[1]: https://eukaryotewritesblog.com/2021/05/02/theres-no-such-th...
If your field re-defines a commonly used term to mean something very precise, obviously you're going to encounter problems when you try work backwards. "Tree" in the meaning of "some phylogenetic categorisation that contains elements x, y, z" may not exist in biology, but "tree" in the meaning of "big leafy thing made of wood" exists as sure as anything.
Drawing lines is for scientific language, not general-purpose language.
Depends on whether the bun is cut in half. If yes it's a sandwich otherwise it's a pizza. /s
Arguments about categorizations can be productive in as far as they allow you a glimpse into the thought-process of other people (=
> In the United Kingdom, value added tax is payable on chocolate-covered biscuits, but not on chocolate-covered cakes.[14][15] McVities defended its classification of Jaffa Cakes as cakes at a VAT tribunal in 1991, against the ruling that Jaffa cakes were biscuits due to their size and shape, and the fact that they were often eaten in place of biscuits.[16] McVities insisted that the product was a cake, and allegedly produced a giant Jaffa cake in court to illustrate its point.[16]
> The court discounted the expert evidence, as it went beyond the capacity of an ordinary purchaser.[17] The product was assessed on the following criteria:[18][19]
* The product's name was regarded as a minor consideration.
* The ingredients were regarded as similar to those of a cake, producing a thin cake-like mixture rather than the thick dough of a biscuit.
* The product's texture was regarded as being that of a sponge cake.
* The product hardens when stale, in the manner of a cake.
* A substantial part of the Jaffa cake, in terms of bulk and texture, is sponge.
* In size, the Jaffa cake is more like a biscuit than a cake.
* The product was generally displayed for sale alongside other biscuits, rather than with cakes.
* The product is presented as a snack and eaten with the fingers, like a biscuit, rather than with a fork as a cake might be. The tribunal also considered that children would eat them in "a few mouthfuls", in the manner of a sweet.
The Irish Revenue Commissioners also regard Jaffa cakes as cakes, since their moisture content is greater than 12%. As a result, they are charged the reduced rate of VAT (13.5% as of 2016).[22]
Predictability and testability being the big problems. Science is prediction, not explanation. Psychology has a huge replication problem.
Advertising, though, has become a science. Predictability, testability, controlled experimental conditions, quantifiability - that's Google's business.
You and the person you are responding to are in agreement. They were being sarcastic.
There's not a lot of real predicting in bio research IME (granted only with a couple subfields). A bunch of things are tried and then whatever works is the focus of the publication, with a write up implying there was a clear hypothesis from the start. Of course not all bio labs are like this, but I'd argue not all psych labs are either.
A lot of this is bad incentives and poor training, but human research is especially hard to control. So I think the issue is partially inherent to the field (in both psych and to some extent bio).
I'd guess the replication crisis in biology is not evenly distributed among branches of biology; for instance I guess it's much worse in biomedicine than it is ornithology. And even within one branch of the biology, I expect the replication crisis is not evenly distributed. Studies about the behaviors of birds probably reproduce less than studies about the distribution of birds (particularly since the introduction of photography, which removes a lot of ambiguity from the later but not necessarily the former.)
Simply because we can't question the results of the field: medicine, vaccines, etc. Whereas for psychology, I fail to think of a single tangible result.
CA was years ago. Google probably has a much more refined model by now. Too bad they aren't sharing it.
[1]: https://news.ycombinator.com/item?id=27097590
[2]: my comment on [1] https://news.ycombinator.com/item?id=27099032
I'd like to add business (economics) to the list of soft ones as well. They are all, in a way, studies of human behavior.
That is not to say that those soft fields dont apply a lot of scientific methods and use techniques discovered in "hard science" fields (statistics, calculus, etc.)
The problem is, even in the "hard" sciences a lot of research is hardly scientific. When it comes to human behavior it probably could be studied in a more thorough and methodical way, but for now we simply lack the tools. Today's psychology will be considered pseudoscience in the future, same as astrology which comes from a time when we knew nothing about other solar systems.
The issue with psychology is that it’s 1. relatively new 2. Area that is challenging to work in 3. Many in the field have questionable motives and perform work that lacks rigor.
> Happiness research is a great example of why psychology isn’t science. How exactly should "happiness" be defined? The meaning of that word differs from person to person and especially between cultures. What makes Americans happy doesn’t necessarily make Chinese people happy.
I would say that the specific cause may differ but the feeling does not. And even the differences in the causes are exaggerated. Sushi may make you happy. A cheeseburger makes me happy. In both cases, our favorite food makes us happy. Paris makes you happy. London makes me happy. In both cases, our favorite city makes us happy. Our parents fighting would make our childhoods unhappy, though the particular individuals who fight would differ. Interests shared with a mate, studies show, makes a marriage happier, though the particular interests may vary from couple to couple.
Yeah, it's hard to measure concretely. But even the most concrete sciences, like physics and astronomy are squishy, just ask quantum physicists. And I see revisions to biology in the news several times a year.
Just because you can't nail down every last fact, doesn't mean that you can't comb through the foam to extract and build a body of knowledge.
There are areas of psychology that rely on methods that are not scientific, because their subject of study is too hard to approach in a rigorously scientific way (though that doesn't mean it's not legitimate knowledge creation, it just comes with more caveats).
But there are also areas of psychology that can be, and are studied with scientific rigour, such as with similar experimental designs to biology.
As someone who has formally studied psychology, the main impression this article makes is that the author knows very little about the contemporary academic discipline of psychology, and is just a bit arrogant about it.
1. https://en.wikipedia.org/wiki/Demarcation_problem
> The demarcation between science and pseudoscience is part of the larger task of determining which beliefs are epistemically warranted. This entry clarifies the specific nature of pseudoscience in relation to other categories of non-scientific doctrines and practices, including science denial(ism) and resistance to the facts. The major proposed demarcation criteria for pseudo-science are discussed and some of their weaknesses are pointed out. In conclusion, it is emphasized that there is much more agreement on particular cases of demarcation than on the general criteria that such judgments should be based upon. This is an indication that there is still much important philosophical work to be done on the demarcation between science and pseudoscience. [2]
2. https://plato.stanford.edu/entries/pseudo-science/
Obligatory XKCD: https://xkcd.com/435/
At the end of the day, what counts is whether the experiments confirm the theory or not. Practically speaking, in the 'applied-psychology' field of psychotherapy there is at least one method, CBT, that has been battle tested and is widely used for curing various disorders such as fobias. From the scientific point of view, no psychotherapeutic study will ever be classified as scientific as it's inherently impossible to conduct a double-blind study (single-blind is the maximum you can get). Nevertheless, for all intents and purposes, CBT is working and is helping people statistically more than a simple talk with a friend or religious rituals (which also have their efficacy levels).
Granted, "For the past few months I felt bad, but now I feel quite good" is not very scientific, but has enormous value for people who are being helped in this way. Instead of stigmatizing psychology as non-science, we should promote approaches that favor reproducibility and proved to be efficient in practice.
Really very sad. Just because psychology (and economics) is not science, does not mean they are useless or untrue. Science is awesome, but not the only path to truth.
What other way meaningfully leads to the truth?
IMO it is a decent question.
I was careless in my use of the word truth. I mean "useful information that can be relied on". I do not mean anything supernatural.
A lot of human knowledge that fits that category does not come from science. Most medicine, for instance. It comes from the experience of clinicians applying their practice and sharing their knowledge.
"five basic requirements for a field to be considered scientifically rigorous: clearly defined terminology, quantifiability, highly controlled experimental conditions, reproducibility and, finally, predictability and testability." says the article.
The truth (from economics) that all else being roughly equal increasing price reduces demand cannot be found under those restrictions.
Nor could astrophysics.
String theory deals with measurements that are mind-boggling small to the point where experimentation is impossible. String theory cannot be tested in a lab or in any sort of tactile experiment. Experimentation is a critical piece of a science and so since String Theory cannot be tested with the scientific method, all of physics is not a science.
Of course that argument is absurd and so is this article.
Psychology fulfills all of these criteria. Yes, happiness is subjective, but there is systematic variance that can be accounted for. Psychology just tends to study a lot of complex behaviours that have to be measured indirectly, either through self-report or behaviour. But there's still systematic variance, and psychology uses theory to explain that variance using underlying causal mechanisms. Nothing in here is any different than other sciences. Scientists study plenty of things that are latent and cannot be directly observed. That's part of what makes science so powerful -- being able to incorporate those aspects that can't be directly observed into a theory that encompasses both the unobserved and the observed, and explains their relationships with each other.
- Cargo Cult Science, by Richard Feynman (1974) <https://calteches.library.caltech.edu/51/2/CargoCult.htm>
The author makes it sound like you just pull a scale out of your ass. There are processes that researchers must follow to create a scale to measure something like happiness. If these rules are not followed correctly, you won't get your research published in reputable journals.
The only measures I’ve heard of are not standardized and typically self-reported / survey based measures.
1. Neither is the entire field of mathematics (or philosophy or logic or history, you name it). Not everything needs to be a science (in the "empirical" sense) to have value or some claim to truthiness/rigor.
2. Or "field X doesn't have the perfect predictability/reproducibility of a proper hard science." Sure. But neither does my doctor, and yet the challenges in the practice of applied medicine don't invalidate the entire science of biology. Which pill will cure me, how long do I have to live, what do you think my symptoms mean--will get you an "I don't know but here's my best guess..." from any honest doctor. Turns out it's hard to go from the micro level (say a cell or single organ) to the macro level (the body as a complex biological system interacting with the environment) in any field. And some fields operate entirely at the macro.
Systems that are sufficiently abstract, interactive, adaptive, subjective, or social tend to be harder to make generalized conclusions about, compared to something you can observe in a vacuum or isolate under a microscope or put in a double blind experiment.
In some cases, maybe we're just not at a point yet where we can make super strong claims. How young is the field of psychology? Or of economics? Think back to Freud and the whole pseudoscience of psychoanalysis, and then look at how far we've come. We've progressed on a hard problem the way chemistry has moved on from alchemy.
The challenges inherent in the social sciences shouldn't mean we just throw up our hands and give up. We try to make them more scientific, develop new standardized measurements for them. Happiness is harder to quantify than mass. Should we not bother trying? Perhaps we never get to the point of perfect predictability. Living organisms are hard to predict, much less entire systems of intelligent ones. But we can learn at least something.
-Allen Frances
When something is categorized as "science", you make assumptions about it. Categorizing psychology this way is misleading, and devalues the categorization's purpose.
This piece seems to criticize just one subset of psychology - happiness research - and use that to dismiss the entire field. I feel like writing putting one discipline above another should be a bit more vigorous in that dismissal, even if I ultimately agree with the general sentiment.
> Like what you’re reading? Too bad because here is a SUPRISE MODAL CALL TO ACTION, sucka
Are my ego and id in contention? Is Mars in retrograde? Same effect on my mental health.
and here's the real article posted on https://www.realclearscience.com/blog/2012/07/keep-psycholog...
https://www.motherjones.com/politics/2013/10/american-counci...
This article was written just to troll up attention for his book that came out the same year (2012) with a title that makes the bias more obvious: "Feel-Good Fallacies and the Rise of the Anti-Scientific Left."
I've never heard of the guy before, but it only took a minute of googling to find that out.
I recommend people try the "lateral reading" technique when you come across articles like this. Essentially, Google the source to see if it's biased and unreliable like this is. More on lateral reading and the SIFT technique (stop, investigate, find, trace) https://library.nwacc.edu/lateralreading/sift
“ Correlations are a profound part of our universe.
When observing the universe, humans can never prove any facts about the universe. We can only establish correlations. Correlations between events that occur in our universe is the furthest "truth" we can establish about the universe short of a full on proof.
What this means is that nothing in the physical universe can be proven. Proof is the domain of maths and logic, correlations is the domain of science. Science cannot prove anything, it can only establish correlations and causations.
The reason this occurs is because at any time in the future one can observe an event that contradicts a hypothesis. You can hypothesize that all birds have wings and observe 2 trillion birds with wings but you never know when one day you'll observe a bird without wings disproving your entire hypothesis. That is why nothing can be proven, you can only correlate things through observation.
The other interesting part about correlation is what it isn't: Causation. People often talk about how correlation is not causation but people never talk about what causation is and how to establish it. If I can't empirically use correlation to establish causation how on earth is causation ever formally established? People rarely question this disconnect.
The fact is, causation is rarely formally established but a method does exist and it's subtle.
If I observe that whenever Bob flicks a switch the light comes on then I established that the light coming on is correlated with Bob flipping the switch. This is as far as I can go with just observation.
To establish causation I must make myself both an observer and an entity that is part of the system itself. I have to take control and flip the switch randomly and observe that when I don't flip the switch nothing happens and when I do flip the switch the lights come on.
By doing this I establish causation. To establish causation to higher and higher degrees I need to Cause (keyword) random events and make sure that a cause influences an effect AND absence of a cause and therefore absence of an effect occurs.
Also note that establishing causation is not proof. At any point in time in the future I can flip the switch and the light may not come on which is contradictory evidence for causation.
Causation in the statistical sense is like correlation, you establish it to a degree of confidence but you can never Prove A caused B.
Note that there are ways to test causal hypotheses without intervening.
For example, suppose we wish to test the hypothesis that smoking causes lung cancer via the main mechanism of tar buildup in the lungs, against the alternative hypothesis that smoking is correlated with lung cancer because of a gene that predisposes people to both smoking and lung cancer (this example comes from Judea Pearl’s Book of Why).
If the former hypothesis is true, then we should see a correlation between smoking behavior and tar deposits, and we should also see a correlation between tar deposits and lung cancer even after controlling for smoking behavior. Composing the causal effects at each stage, we can then calculate the indirect causal effect of smoking on lung cancer. If this suffixes to explain the correlation, then that rules out the alternative genetic explanation for the correlation.
Of course, we can always propose ad-hoc further hypotheses that complicate the analysis: maybe the correlation between smoking and tar deposits is itself non-causative, etc.
This only goes to show that scientific inquiry must be done with judgment and with expert domain knowledge, testing plausible hypotheses in good faith. But that’s true for detecting mere correlations too — we can always doubt our instruments, or claim the data is a statistical fluke. And it’s true in randomized controlled trials: it’s conceivable we didn’t properly randomize, for example.
* This (causation as correlation) revelation was highlighted by Hume and this and other work by him had profound influence on Kant (famously awaking him from his "dogmatic slumbers") and scientists like Darwin and Einstein - the latter obviously in a more healthy scientific age when those at the forefront of physics were not so disdainful of philosophers.”