Daniel Kahneman calls for ‘daisy chain’ of psychology replications
blogs.discovermagazine.com
blogs.discovermagazine.com
http://www.nature.com/news/nobel-laureate-challenges-psychol...
The actual article is worth a read, I was a little intimidated after reading "here’s some overmatter that didn’t make it into the piece because of length" but the actual article is only 712 words. The over-matter is 500 words.
More background about the problem:
"Replication studies: Bad copy" http://www.nature.com/news/replication-studies-bad-copy-1.10...
We say that a study "succeeds" when it illustrates a novel behavior and "fails" when the expected behavior doesn't happen.
This is a very slippery use of words.
A finding that can't be replicated isn't a finding. It's a rumor, an urban myth, but not a scientific step forward.
A study that "fails to replicate" a previous finding isn't a "failure" -- it succeeded at debunking a false belief! And it should be granted wide access to publishing.
(Same thing happens in medical studies BTW, where it's arguably much more dangerous).
Yep. the problem is that to the "laymen", it seems a bunch of people wasted time trying to re-do what someone else has "already" done, and failed. Whether the failure was due to incompetance, chance, or an actual failure to replicate (meaning it is a significant), to the mind of the laymen, they are one and the same. this then end up being difficult for people who might fund such studies to fund these sorts of studies (despite it being quite useful), and leads to people seeking "newer" things to research.
its a bit of a chilling effect really.
Results of failure are still results, and they represent valuable knowledge. Too bad the people who got them consider them a failure.
What would be great: a centralised place for people to share their attempts to replicate a work. If several groups have failed, and nobody has succeeded, then there's certainly room for a rigorous, joint debunking of the experiment. If, as is often the case, there's some subtlety to get the experiment right, then the discussion around it between groups actually enhances the original work.
Not necessarily. How would you repair a study that requires strict controls but human subjects, without violating ethical constraints? You can't. Some studies are perpetually hobbled by the fact that you can't put human subjects in rat cages and force-feed them.
My point is that, beyond the replicability issues discussed on the linked article, there are serious and ongoing obstacles to rigorous experimental work in psychology.
Yes! I would add that failing to replicate existing results is the only interesting outcome of a replicating study.
If we try to replicate a study and arrive at the same conclusions, we have learned little (the original result is a little stronger but that's about it).
If however, a series of new studies fail to replicate original results, we have learned a lot -- we have learned that those original results were false.
I think the whole problem is, ironically, psychological in nature. Results of studies become articles of faith; people identify with their beliefs and hate to admit they have been fooled...
Exactly, and that's precisely the opposite of what science is. We should be excited, not intimidated, by falsifications and getting results we didn't expect.
BTW, I come from the field of robotics and there, hardly any published result is replicated either.
Not at all ignorant. It is a given that a replication should be carried out by a disinterested party, one neither an advocate of, nor critical of, the original work.
Studies are supposed to be repeatable. But in practice a lot of research never gets repeated. I'm sure he must feel a little insulted to have his life's work called into question, even if the cause is other investigators who were fraudulent or incompetent, or even if the questions are arising only indirectly because of unrelated research that happens to use priming.
IANAS, but psychology is obviously not the hardest of the sciences. I think in other areas like e.g. chemistry and biology, it's more difficult to pass off methods and results that are not repeatable _if_ they are going to be foundational and used for lots of future research by others. If some other lab can't get the right results using your methods, I would think they will make some noise. They might first think they are themselves at fault, but they would probably make contact with others outside their lab and make it known, in some way, that these methods were not working for them. Whereas I can't see that as being as common in psychological research. It's too easy if some method is not working to just "make it work". ;)
A welcome and overdue call to action. But I predict the challenge will be ignored, because the practitioners of this psychological specialty know the effort will fail, and that failure will destroy what remains of their credibility.
Public perception doesn't matter much for research in the field, but it doesn't help credibility.
Emotion and personality are valid subjects of scientific study as well. There has been a lot of unscientific opinion on those subjects (Freud being a good example), but that doesn't mean there isn't good science done there as well.
Only if experimental results can be obtained using objective scientific methods, with control groups and strict experimental protocols, and only if the results can be successfully replicated. In other words, they cannot be studied scientifically. That's the meaning of the linked article: no replication. The article doesn't say such studies can be replicated, it discusses how to deal with the fact that they cannot.
No, I am addressing the question, which is whether present psychological research is or is not scientific.
> It absolutely is possible to have well-designed studies that address these topics.
Not ethically, no -- not possible. Prospective double-blind studies are virtually unheard of in psychology because of ethical issues.
> It's how we know anything about depression, anxiety, happiness, motivation, etc. etc.
Yes, true, which is why, in a scientific sense, we don't know anything about these topics. For example, it has recently been determined by way of a meta-analysis that the many antidepressant studies, taken as a whole, contradict each other, and the conclusion is that antidepressants don't work:
Title: "Initial Severity and Antidepressant Benefits: A Meta-Analysis of Data Submitted to the Food and Drug Administration":
Link: http://www.scientificamerican.com/article.cfm?id=faulty-circ...
A quote: "Meta-analyses of antidepressant medications have reported only modest benefits over placebo treatment, and when unpublished trial data are included, the benefit falls below accepted criteria for clinical significance."
In layman's language, "there is no scientific evidence to support the conclusion that antidepressants actually do anything."
The above study did something quite clever -- it combined studies that the drug industry published, with studies that were paid for and conducted but then suppressed. The outcome for this combination, i.e. all such studies, is that there is no meaningful statistical difference between an antidepressant and a placebo.
Why do you believe this to be the only valid form of scientific study? Do you hold all disciplines to this standard?
>Yes, true, which is why, in a scientific sense, we don't know anything about these topics.
We know that certain assessments reliably indicate the presence of depression. We know certain conditions reliably induce depression. We know certain therapies reliably relieve depression. We know certain risk factors lead to greater likelihood of depression.
>For example, it has recently been determined by way of a meta-analysis that the many antidepressant studies, taken as a whole, contradict each other, and the conclusion is that antidepressants don't work.
>In layman's language, "there is no scientific evidence to support the conclusion that antidepressants actually do anything."
Can you explain to me how science can decide one way or another on matters concerning depression if such things are inscrutable to the scientific method?
If a particle physicist discovers something basic in his own specialty, it affects all of physics -- for example, the mass of neutrinos or the presence or absence of a Higgs boson. Cosmologists must pay attention to particle physics, even though particle physics and cosmology would seem to be spectacularly dissimilar.
Such a scientific theory is a two-way street. If cosmology comes up with something like dark matter or dark energy, based on solid observation but not explained, this affects particle physics, and work begins immediately to explain what has only been described. The reason is that the central scientific theory, that defines physics, also unites it.
Psychology doesn't have such a theory -- there is no overarching theory that unites the 53 subfields recognized by the APA, which consequently operate independently of each other and of any serious concern about theoretical implications.
So when you say that there is real science taking place in some of psychology's subfields, you're absolutely right. But if someone wants to imply that this means psychology is a science in the same way that physics or biology are sciences, they're absolutely wrong.
> to dismiss all of psychological science as illegitimate is to ignore a tremendous amount of progress in understanding and human well-being that has come in the past century.
When Michelson and Morley falsified the ether theory in 1887, every living physicist knew exactly what that result meant -- it pulled the rug from under electromagnetic theory and left it unexplained. No one tried to say, "but those other physicists are still doing good work." That's not how science works.
I will know psychology as a field has reached the threshold of science when psychologists courageously propose a central, unifying, falsifiable theory, amenable to practical test that, if it fails, invalidates the entire field. And I won't hold my breath for that development.
Physics has such a theory. Biology has such a theory. Geology has one. Any of those theories can be tested -- indeed, is regularly tested --and the failure to falsify those theories is the only reason for the standing of those fields as sciences.
Science is not scientists working in isolation, in a theoretical vacuum, on studies that cannot be replicated, as with psychology. Science is instead the building of a defining theoretical structure, constructed on a scaffolding of evidence.
> and human well-being that has come in the past century.
That claim is completely falsified by the recovered memory therapy debacle of the 1990s, which singlehandedly erased any public goodwill toward the field of psychology and sowed an atmosphere of distrust that will require many years to dissipate. And recovered memory therapy is just one of many examples in recent times that reveal the danger of not shaping a guiding theory that simultaneously informs research and controls the behavior of clinicians.
Psychology can be about discovering the truth about people's minds on a high level, so it can be a science. It probably hasn't been going about it the right way so far, though. I think most of the field is sitting around theorizing (like the Greek philosophers) rather than come up with testable claims.
We don't get to pick any definition of science we like. Science is by definition an agreement between people, a consensus about what evidence means, what falsifiability means, and finally what science itself means. Everything else is post-modernism.
> Science is anything that searches for the truth in some matter.
That is precisely not how science is defined. Science is not a search for truth, and scientific theories never become true. Some become false, none become true, ever. All scientific theories are perpetually open to falsification by new evidence.
> For example, math isn't a science, because it isn't searching for any truths.
You just picked the only science that violates the general rule, that is able establish something as true, beyond falsification. So your claim about mathematics is exactly, precisely -- perfectly -- wrong.
> Psychology can be about discovering the truth about people's minds on a high level, so it can be a science.
You've just described religion, but science isn't religion. Science is based on skepticism and doubt, not on a mystical search for truth.
Indeed, and the definition of science doesn't contain "must have a unifying theory" as a prerequisite.
> Science is not a search for truth, and scientific theories never become true.
Scientific theories may not become true, but science absolutely is a search for truth. There is a truth, and we're trying to get to it with rational means. I also notice you haven't offered an alternative to my definition, you merely disagreed ("Science is not a search for truth") and argued against a straw man ("scientific theories never become true").
> So your claim about mathematics is exactly, precisely -- perfectly -- wrong.
How so? You just said that mathematics is a science that violates the general rule, even the definition you hint to (falsifiability), and then claimed that this makes mathematics the most pure science there is ("your claim about mathematics is exactly, precisely -- perfectly -- wrong").
> You've just described religion, but science isn't religion.
No, I didn't. I described a way psychology could be scientific, i.e. by experiment.
You misrepresent half of my claims and hand-wave the other half. I didn't expect you, of all people, to do that.
On the contrary. Crafting and then testing theories is at the heart of how science is defined. A theory is an attempt at explanation, and all scientific theories can be falsified in principle.
No theory, no falsification -- but also, no science.
> Scientific theories may not become true, but science absolutely is a search for truth.
Science is not in any way a search for truth. Truth is not a goal in science, only theories that resist falsification. Any scientific theory is perpetually open to falsification -- that is not a property of truth. Truths aren't falsified later as new facts arrive -- that property is limited to theories.
> I also notice you haven't offered an alternative to my definition, you merely disagreed ("Science is not a search for truth")
No alternative is necessary --the claim is false, period. It's easy to explain why (theories must remain open to falsification by new evidence) but there is no reason to offer an alternative, a replacement.
> and argued against a straw man ("scientific theories never become true").
That wasn't a straw man -- it replies to your claim "Science is anything that searches for the truth in some matter." That is a word-for-word quotation. And it's false -- science's goal is limited to finding evidence to support a theory about reality, prevent it from being falsified.
> and then claimed that this makes mathematics the most pure science there is ...
I never said this, anywhere. Those are not my words.
> I described a way psychology could be scientific, i.e. by experiment.
The topic is not whether psychology cold be scientific, if for example ethical constraints were ignored. The topic is whether psychology is scientific.
> You misrepresent half of my claims and hand-wave the other half.
I misrepresented nothing, instead I quoted you word for word, just as I am doing now. You, by contrast, have invented positions for me, like "... and then claimed that this makes mathematics the most pure science there is ..." as though I had taken that position or used those words.
You said:
> there is no overarching theory that unites the 53 subfields recognized by the APA
I said that there doesn't need to be a unifying theory for a field to be considered a science. "One unifying theory" and "a field united by theories" are two different things. You're misrepresenting my words again, even after quoting them.
> Truth is not a goal in science, only theories that resist falsification. Any scientific theory is perpetually open to falsification
You seem to think that, since theories can be falsified, they have nothing to do with the truth. This not only doesn't follow, but is false. The truth is that gravity can be modeled with some equations. Newton's theory was pretty close to the truth, Einstein's is even closer. I have no clue how you can say that science doesn't care about the truth and building an accurate model of the world. Phrases like "the map and the territory" come to mind here.
> No alternative is necessary --the claim is false, period.
So you don't have a definition of science, you only have disagreement to other definitions? That doesn't sound very informed.
> It's easy to explain why (theories must remain open to falsification by new evidence)
Who said anything about theories being confirmed? If objects fall a certain way, they fall a certain way, some models are more correct than others, even if maybe none are completely correct. Science tries to build as accurate models as it can, getting closer to the actual truth of the world. Where's the problem here?
> And it's false -- science's goal is limited to finding evidence to support a theory about reality, prevent it from being falsified.
Yes. What you call "reality", I call "truth".
> I never said this, anywhere. Those are not my words.
Then I misunderstood. You said that my claim that mathematics is not a science was "exactly, precisely, perfectly" wrong. Mathematics isn't a science by your definition or mine.
> The topic is not whether psychology cold be scientific, if for example ethical constraints were ignored. The topic is whether psychology is scientific.
No, the topic is this claim you made:
> I will know psychology as a field has reached the threshold of science when psychologists courageously propose a central, unifying, falsifiable theory, amenable to practical test that, if it fails, invalidates the entire field.
I completely disagree. They can have two, three, or a hundred theories like that, and still be a science. The one unified theory isn't special, and that's what I'm arguing.
> I misrepresented nothing, instead I quoted you word for word, just as I am doing now.
And then misrepresented the meaning of the words you just quoted, as above.
> You, by contrast, have invented positions for me, like "... and then claimed that this makes mathematics the most pure science there is ..." as though I had taken that position or used those words.
I misunderstood one of your (unfounded) disagreements, hardly an indictment.
But that's false. All scientific fields have testable, unifying theories -- single theories that all practitioners agree define their field (even if they plan to falsify the theory) -- and that defines them as practitioners.
You name the field, and I will tell you the single, unifying theory that defines the field. Only one requirement -- you have to choose a science, not a pseudoscience.
> "One unifying theory" and "a field united by theories" are two different things.
Yes -- one describes to the cause, the other describes the effect. Both refer to the requirement that a scientific field be defined by a unifying theory, a theory that makes physics physics, or biology biology. A theory that successfully distinguishes astronomy from astrology. In other words, you're splitting hairs.
> You're misrepresenting my words again, even after quoting them.
I did nothing of the kind, and I just proved it.
> You seem to think that, since theories can be falsified, they have nothing to do with the truth.
That is not something I seem to think, that is a fact. Truth cannot be falsified -- that is not how truth is logically defined. A tentative truth is not a truth. A potentially falsifiable truth is not a truth (that's a theory).
> The truth is that gravity can be modeled with some equations.
This deliberately conflates a statement about gravitational equations with a statement about gravitational theory. If you had instead said "gravity can be accurately modeled with some equations" that would be different -- very different. The first makes a factual statement about an equation, the second makes a theoretical statement about physics.
> So you don't have a definition of science, you only have disagreement to other definitions?
I defined science based on its requirements, requirements accepted by all scientists and granting agencies. Science is even defined in courts of law at times, as in this example:
http://www.talkorigins.org/faqs/mclean-v-arkansas.html
The above legal ruling, meant to keep Creationism out of the classroom, defines science this way:
1. It is guided by natural law;
2. It has to be explanatory by reference to natural law;
3. It is testable against the empirical world;
4. Its conclusions are tentative, i.e. are not necessarily the final word; and
5. It is falsifiable.
So you see, I didn't invent this definition -- it is one on which scientists agree, even those who oppose science on philosophical grounds.
>> I never said this, anywhere. Those are not my words.
> Then I misunderstood.
Misquotations don't arise from misunderstandings, and all such cases can be resolved by using only actual quotations -- by using copy, paste and no creative typing.
>> I will know psychology as a field has reached the threshold of science when psychologists courageously propose a central, unifying, falsifiable theory, amenable to practical test that, if it fails, invalidates the entire field.
> I completely disagree. They can have two, three, or a hundred theories like that, and still be a science.
1. You just declared astrology a science. Congratulations. By your reasoning, astrology is part of astronomy, differing in only a few theoretical details. But to a scientist, astrology is not astronomy because it has a different theory.
2. Do you know why cellular biology is not particle physics? Different theories.
3. DO you know why particle physics and cosmology are both accepted as part of physics? Same theory.
4. Do you know why physics is accepted as a science? It's because all practitioners of physics, from theoreticians to engineers who build bridges, all agree on the single theory that defines what they do, and that defines physics and distinguishes it from other scientific fields as well as unscientific fields.
If a doctor claims that shaking a dried gourd over a cold sufferer will eventually make the patient better, and claims the reason is the gourd treatment, he has crafted a theory, but it's not a medical theory -- it contradicts the prevailing theory that defines medicine.
The dried-gourd practitioner is excluded from medicine because his theory doesn't conform to the defining theory of the field. That by itself is sufficient reason to have a single theory (there are other equally important reasons).
> And then misrepresented the meaning of the words you just quoted, as above.
You failed to make your case.
> I misunderstood one of your (unfounded) disagreements, hardly an indictment.
Your invented quotation wasn't a misunderstanding, it resulted from your unwillingness to quote only my words directly, using the traditional and undemanding copy, paste meme.
Correlation isn't causation, the unifying theory isn't what makes the field scientific.
Are you saying that Kahneman, or Milgram, aren't scientists? Since they used the scientific method, that makes them scientists in my book. What's your objection?
> Truth cannot be falsified -- that is not how truth is logically defined. A tentative truth is not a truth. A potentially falsifiable truth is not a truth (that's a theory).
That is correct. I never said that a theory is truth, or that truth is a theory. Theories are what we speculate the truth to be, until we are proven wrong. I don't know how anyone can be arguing against this, it seems very fundamentally obvious to me.
> That is not something I seem to think, that is a fact.
No, it's not a fact. A theory has as much to do with the truth as a map has to do with the actual territory. Saying that maps have nothing to do with the territories they try to describe is ridiculous. The fact that your map may be inaccurate doesn't mean the territory doesn't exist.
> So you see, I didn't invent this definition -- it is one on which scientists agree, even those who oppose science on philosophical grounds.
And I agree with it too. That definition is in no way at odds with what I'm saying.
> Misquotations don't arise from misunderstandings
I didn't quote you.
> But to a scientist, astrology is not astronomy because it has a different theory.
> > I completely disagree. They can have two, three, or a hundred theories like that, and still be a science.
> You just declared astrology a science. Congratulations.
How does that even follow? I declared astrology a science because I said it doesn't need a unified theory? You're denying the antecedent:
A science doesn't have to have a unifying theory to be a science. Astrology doesn't have a unifying theory. Therefore, astrology is a science.
> By your reasoning, astrology is part of astronomy, differing in only a few theoretical details.
"By your reasoning, chemistry is part of physics, differing in only a few theoretical details." What it's called and whether it's a science are two different things.
> But to a scientist, astrology is not astronomy because it has a different theory.
Astrology isn't astronomy mainly because astrology is not a science, since it doesn't care about testing its claims.
> Do you know why physics is accepted as a science? It's because all practitioners of physics, from theoreticians to engineers who build bridges, all agree on the single theory that defines what they do, and that defines physics and distinguishes it from other scientific fields as well as unscientific fields.
So how is it possible for both physics and chemistry to be sciences? If you can have many unifying theories, then this just becomes an issue of labels, and I'm explicitly not discussing labels, but whether or not you need a single theory in order to be scientific.
Physics and chemistry don't have a single unifying theory, yet they're both sciences. That's because a unifying theory is not what defines science. It may be what defines a field, which is why physics and chemistry are different, but a science is defined by following the scientific method, i.e. a search for truth.
EDIT: Goddamnit, this must be the longest argument between two people who agree ever written. I can't believe we can't even communicate something this simple efficiently. I don't think we've disagreed on a single thing all this time.
And you should read a basic primer in psychology. You claim there have never been "central, unifying, falsifiable" theories that are "amenable to practical test" that have the potential to fail. I guess you've never heard of behaviorism or cognitive psychology. Evolutionary theory and computational models are also important components of psychological research.
On the contrary, I know this historical period, and this experiment, very well -- certainly better than you do.
> When some physics experiment comes out, "every living physicist" doesn't immediately know "exactly what that result meant."
I never said what you claim -- you have above invented a quote to argue with. As to what I did say, over time, of course they do, and it was certainly true about Michelson-Morley. The result caused an effort to, as was said at the time, "save the ether", an effort that extended from 1887 to the time of Special Relativity, which is what resolved the crisis. Details here:
http://en.wikipedia.org/wiki/Luminiferous_aether
> It not nearly as rational and positivist as you would think.
That is exactly what it was, and I just showed this by providing a link to a detailed history. I recommend that you read the history before pontificating about it.
> Science is a human activity.
Yes -- it's how we deal with human foibles, with our tendency to believe things for which there is no evidence. As you have just done with the content of my post, which doesn't have the words in it that you falsely claim to be present.
> You claim there have never been "central, unifying, falsifiable" theories that are "amenable to practical test" that have the potential to fail.
And? Prove me wrong. There is no central, unifying, testable, falsifiable theory in psychology, that would be to psychology what the Standard Model is to physics -- something that unifies all theoretical and experimental work as well as practice.
When a bridge or airplane is built, it must conform to physical theory or the designer is likely to be prosecuted or expelled from the field. The reason? There is such a theory, it is very reliable, and it informs all research and practice -- from theoretical work to engineering.
Prove that this exists for psychology. Prove that recovered memory practitioners, or facilitated communications practitioners, or those who offered treatment for the now-abandoned "disease" called Asperger's, were prosecuted because their activities failed to conform to the accepted, unifying psychological theory that is to psychology what physical theory is to physics and biological theory is to biology.
> I guess you've never heard of behaviorism or cognitive psychology.
I guess you don't realize that these subfields don't have testable, falsifiable theories, and they certainly don't possess a theory that unifies the entire field as physics and biology theories do.
> Evolutionary theory and computational models are also important components of psychological research.
Yes, but these aren't scientific theories -- they lack the properties of testability and falsifiability.
If psychology were to acquire a central, unifying theory, the defining trait of all legitimate sciences, clinicians would have to conform to it, something that is spectacularly false right now, and will remain so for the foreseeable future. As thing stand, clinicians can do whatever they please -- they can invent new diseases (Asperger's, recovered memory therapy and facilitated communications are recent examples) that exist until public outrage requires their abandonment. Imagine if doctors had the "freedom" that psychologists have -- it would challenge the scientific standing of medicine.
It certainly challenges the scientific standing of psychology, to the degree that the present director of the NIMH has recently diplomatically called for the wholesale replacement of psychological methods with neuroscience:
http://www.scientificamerican.com/article.cfm?id=faulty-circ...
A quote: "In most areas of medicine, doctors have historically tried to glean something about the underlying cause of a patient’s illness before figuring out a treatment that addresses the source of the problem. When it came to mental or behavioral disorders in the past, however, no physical cause was detectable so the problem was long assumed by doctors to be solely “mental,” and psychological therapies followed suit. Today scientific approaches based on modern biology, neuroscience and genomics are replacing nearly a century of purely psychological theories, yielding new approaches to the treatment of mental illnesses."
> And you should read a basic primer in psychology.
Do I need to read a basic primer on astrology before describing it as pseudoscience? No, not really -- to accurately identify it as pseudoscience doesn't require that level of familiarity. All one need do is read its history, read its longstanding contempt for scientific and experimental discipline. You did notice, didn't you, that this is the exact point made by the linked article? The article in which the Nobel Prizewinner says, "Your field is now the poster child for doubts about the integrity of psychological research… I believe that you should collectively do something about this mess."
Or didn't you get the memo?
Let's face it: Physics is easy! It's objects are often cheap to study, and usually doesn't react to the behavior of the observer. Additonally, the observed behavior is often regular and easy to model with maths. There are millions of research grants, and job opportunites because its findings are often economically useful. Also, people in general don't think they 'know' its findings in advance, so it's much easier to overcome your own biases, and opinions. In most cases, there is often no politics involved about the results.
In contrast, the social sciences are really hard. Lots of causes make research harder: Objects often react to the researcher, or the setup. They want to get payed for their time. Sometimes, they react to the experimental setup due a political stance. There are ethical constraints for experiments. The shere variability of human behaviour makes it hard to use mathemathical models to describe theories. Its findings are not as useful economically, because they cannot be impletemented on a large scale. Potential users of the research often think, they 'know' how people react, anyway (folk psychology). For some findings, many people are also motivated to question the results because it conflicts with their political opinion.
Overall, the barriers are much higher for the social sciences to make progress. I think these fields deserver the label 'hard', for finding the truth is really hard here.
Psychology is not dismissed as a science because it's complicated, it's dismissed because it's not science. The fact that it's complicated is irrelevant to its standing among sciences.
And offering the explanation that it's comparatively complicated fails any test of common sense -- remember quantum theory? It's more complex than any psychological theory, and yet we acquire perfectly reliable results to ten decimal places. In fact, quantum theory is the single most successful scientific theory in existence, yet no one fully understands it. "Anyone who is not disturbed by quantum physics has not understood it." -- Neils Bohr.
> Let's face it: Physics is easy!
Only to those who don't understand the subject.
> In contrast, the social sciences are really hard.
Psychology is not a science because psychological research is so difficult? Okay, but if I were a psychologist, I would ask you not to be on my side.
> ... for finding the truth is really hard here.
Science is not about finding truth, and scientific theories never become true. Some of them resist falsification for extended periods, but all of them are perpetually falsifiable in principle by new evidence.
On the first point, I was addressing the fact that some areas of research within psychology are amenable to objective, empirical observation, whereas others are not. I was not defending the philosophical grounds for classifying the entire field of psychology as scientific. I realize that even the "hard" subfields of psychology do not necessarily stand up to certain philosophies of science in the way that physics does. But I am a pragmatist. The scientific method, to me, is systematic observation and reasoning about the world that produces useful results[1]. Psychology would more accurately be called "academic study" of behavior, or something like that, but I feel "science" is a term that can be used in everyday language more loosely than in a strict philosophical argument, which I hadn't intended to be making. Just like when a scientist says "the Higgs Boson exists" when they should really say "evidence suggests at 5-sigma there is a subatomic particle with a mass of approximately 125 GeV." It's semantics, and that's where our disagreement starts. I concede, philosophically, you are correct about what is or isn't science.
On the second point, I believe a major problem with "psychology" is that it is too big of an umbrella term. That is why I defend its nuances against criticisms of "psychology is not legitimate science."
>That claim is completely falsified by the recovered memory therapy debacle of the 1990s...
For you to "completely falsify" the claim that psychological inquiry has increased human well-being because there are examples of people getting something wrong is bullshit, and I think you know that if you give it a second thought. I won't claim implementation of mistaken (due to it's non-scientific nature) psychological understanding never does harm, but on net, our understanding of psychology is absolutely beneficial in many domains, even if you don't want to call it science.
[1] I would include, of course, production of knowledge that does not have immediate application.
Skinner and Pavlov quantified pretty important systems and heuristics your grey matter works on, and are directly responsible for the underlying systems being applied in Zynga - heck - ALL video games.
Take a lot of psych as trying to figure out which underlying subsytems are active when people make decisions.
Even Freud's understanding that most of human calculation occurs in the subconscious was a fundamental jump forward for humanity - and IMO, it put a stop to the belief of rationalty which underpinned a good part of the enlightenment. (bit hard to encourage people to be rational when their wet ware isn't capable of handling it.)
Yes at the same time it can be very handwavy - and some of their best measurements are of dynamic systems in flux.
http://chronicle.com/blogs/percolator/building-the-perfect-j...
A local friend (a mathematician who works as a staff researcher on human behavioral genetics research) and I will be in the weekly journal club meeting later today at the University of Minnesota journal club. This week's topic in the journal club is discussion of Uri Simonsohn, the "data detective." Several of my recent submissions to Hacker News have been of readings related to our discussion today. My local friend sent out this background discussion by email yesterday:
"On Wednesday, September 26, Econ Nobelist and psychologist Daniel Kahneman sent an open email message to a group of social priming researchers and one of them forwarded the message to Ed Yong, the science journalist who interviewed Uri Simonsohn for one of this week's readings. This is that email message:
http://www.nature.com/polopoly_fs/7.6716.1349271308!/suppinf...
"Ed Yong contacted Kahneman and interviewed him, then he wrote this short article for Nature News which appeared Wednesday, October 3
Nobel laureate challenges psychologists to clean up their act: Social-priming research needs “daisy chain” of replication
http://www.nature.com/news/nobel-laureate-challenges-psychol...
"and this entry in his blog:
Nobel laureate Daniel Kahneman calls for ‘daisy chain’ of psychology replications
http://blogs.discovermagazine.com/notrocketscience/2012/10/0...
"The story was also picked up by Chronicle of Higher Education on Thursday night, October 4:
Daniel Kahneman Sees ‘Train-Wreck Looming’ for Social Psychology
http://chronicle.com/blogs/percolator/daniel-kahneman-sees-t...
"In related news, Uri Simonsohn posted his paper on statistical detection of data fabrication on the Social Science Research Network site on July 22, 2012 (draft dated 7/21). He has since posted a revised draft, dated July 29, which can be downloaded from this page: