Hacker News Reader

Comment by p-e-w | Hacker News Reader

> is it common to work with so small sample sizes in psychology?

Yes. Even much smaller sample sizes are common. I've seen "results" published in supposedly reputable journals with sample sizes as small as 9. Not 9 other studies that were meta-analyzed, mind you. 9 individual people.

This is just one of several reasons why most of psychology is junk science, if not outright fraudulent. Statistical methods is usually a required course for students in such disciplines, so I do not buy the "they don't know better" excuse. Not that you need to know statistics in order to understand that you can't conclude anything from a sample size of 9.

View on HN

That is not the reason it is junk. Typically the reason to call something junk is that designing the study itself is difficult, not sample size. There are analysis of small to large groups to determine how small a group can be that will still reflect the whole. 9 does seem on low end, but I think the typically accepted size is something in the 20's of individuals.

I'm guessing that if 9 is enough to call it junk, then 20 would not sway your opinion. What is a good number then? 100? 1000?. Guess could do identical study on groups of 10,100,1000, and show that the results replicate across the size. Of course, this has been done, hence why they get away with smaller sizes.

This isn't a guessing game, really. You have some prior knowledge of the expected effect size - that allows you to calculate the sample size required for your statistical test to have a certain power. That is an essential and very simple part of a good experimental design which, unfortunately, is very often ignored.

There is no fixed size of samples that guarantees a result is statistically significant. How many samples are needed depends on the desired p-value among other things, which in turn depends on prior assumptions, and on the quantities being measured. "Typically accepted sizes" are nothing but cargo cult science.

That's precisely why psychologists take statistics classes as part of their education. I get that those aren't everyone's cup of tea, but people who don't even understand the basics have no business conducting and publishing studies. In fact, well-designed scientific studies typically start with a discussion of p-values, sample sizes, possible sources of biases, etc. If you don't see any mention of those in a study, you know you're looking at junk.

Psychology gets a bad reputation because it is still new. Remember, Astrology turned into Astronomy, Alchemy became Chemistry, and Philosophy (old) became Math and Physics.

Psychology is still a newcomer, and study of the mind and the human animal is difficult because it is complex and hard to measure. That doesn’t mean you can’t start somewhere.

Psychology gets a bad rep because a lot of bad science is conducted in the field. It’s rate of reproducibility is among the lowest of all sciences.

These are structural and methodical issues which are not at all related to it’s age. In many cases it is simply bias, negligence or pure fraud.

You are to quick to blame bad actors. I don't think that the typical student or professor has so much evil intention for negligence and fraud.

These are examples of a 'new' science. Humans are complex and hard to measure, so yes, there are problems with study design, and re-producibility. Because they are working it out. That is because of the field of study, not evil intent, some dark cabal of researchers putting out false data on purpose. If there is some obvious way to solve these problems, then those people should do it.

I was going to throw out some sarcastic example from another field. But guess, when I think about it, maybe it does happen. String Theory in Physics is an example of "group think" leading an entire field down the wrong rabbit hole of stuff that is faulty and can't be measured. So guess it can happen anywhere.

For what it's worth, during my psychology courses in university, I witnessed professors lie to the class, sometimes in response to direct questions. The topics were things that psychology professors would surely know, so I don't think they were simply misinformed.

Being new is not an excuse for producing meaningless garbage.

It's okay to not know. It's not okay to claim results when there aren't any to be drawn from the available data.

And astrology did not "turn into" astronomy. Astronomy developed (in part) as a result of astrologers trying to predict the motion of celestial objects. But astrology is not at all the forerunner of scientific astronomy, since its main concern (how celestial movements influence human lives) lacks any factual underpinnings. It's not that astrology produced incorrect results; the entire endeavor is fundamentally wrong.

You are only looking at other fields successes and only Psychology's failures. Quick to forget all the false starts and wrong directions in other fields.

Might need to do some history readying. Of course the study of celestial movements eventually became astronomy, who do you think was looking at the stars before astronomy existed. All of history isn't some nice linear line of analytic reasoning. I wonder if HN also has a general bias against history, since it is a non-technical field with a lot of interpretations.

> Of course the study of celestial movements eventually became astronomy

Astrology is not "the study of celestial movements". It's the utterly baseless claim that those movements have a high-level influence on human lives. That's not proto-science, it's pseudoscience. And it's not at all comparable to incorrect theories in other fields (like phlogiston or the luminiferous aether). There is no reason to expect that astrology works, and its claims are made without any supporting evidence. That has nothing to do with science.

You are making my argument for me. Do you not see it? Make the leap.

I am NOT saying Astrology is true. I'm saying that a lot of people over history, long before astronomy exited, observed the stars, and traced their movement, and yes, some of them attributed their movements to 'gods', or did what today we called 'Astrology'. Then eventually, someone was like, man, these celestial bodies seem to follow some patterns, and drifted away from thinking it was 'the gods', and started putting math to it, and boom, we look back on it and call it 'Astronomy'.

But Astronomy did not pop into existence fully formed as a science.

I wonder if we have any historical studies or evidence to prove either of these views. Seems if it did or didn't "turn in to" there should have been someone writing about it at the time

Guess that is why I'm surprised this is even an argument. Can do quick search and find many papers and books related to historical developments of the sciences and how in early times they were related. But that would take reading and then trusting historians, which I'm not sure is allowed either these days. In post-truth world, can't trust any studies, can't trust history, can't trust what we see.

Psychology has no "because it is still new" excuse: https://en.wikipedia.org/wiki/History_of_psychology

Electrical engineering, aerospace engineering, nuclear physics, and many other difficult fields of study are far, far newer than psychology. And yet have no problems with reliably producing significant, real-world results.

You are only looking at other fields successes and only Psychology's failures. Quick to forget all the false starts and wrong directions in other fields.

I suggest, instead of bringing up how other science has failed too, you bring up the successes of psychology. Nobody here will be convinced by scientific whataboutism, however, if you were to compose a comment on the important successes of psychology with cited sources i think that would work much better

The antagonism is to great. You want me to cite sources to justify an entire field? Just go look. Why would I spend a lot of time on sources that nobody will read. It isn't my job to educate the world. Do your own research.

Big five personality types, fight or flight brain circuits, pause and plan brain circuits, fundamentals drive of brain to predict what happens next (externally and internally), lack of universal emotional responses. The existence of things going on in the brain that the mind has very limited awareness of (the “unconscious”).

One hallmark of actual sciences is that they notice their false starts and wrong directions, learn from them, improve fairly rapidly, and can ~reliably deliver complex things that actually work.

By that metric, Psychology seems profoundly lacking.

Do I really need to go through and trace the entire history of psychology and every false start and course correction in order to prove that they do it too? Do you see the difference in the burden of proof. You are starting from the position of 'Psychology' is all false, now proof everything by citing everything over again.

But, to do so, I would need to cite something in Psychology, and you can just deny it again. What evidence can be cited if all evidence is dismissed.

You will probably say something along the lines of 'what can be verified or repeated'. Ok, just go look. Why do I need to re-cite an entire discipline, just go read it.

Cool. So when do we get whatever science psychology will turn into?

I think they call themselves neuroscientist now. That seems more respectable, like a real science. I'm sure people will say "no, that is different". But that is what we are discussing, Psychology grows and branches and now there is something called "Neuroscience" that is more refined, and has narrowed the field to something measurable.

Almost. Specific sections of neuroscience, which are actually just cell bio or biochem applied to brain tissue, sure. Often behavioral neuroscience has much of the same problems of not really having good definitions that map to tangible observables. Once good definitions are made, only then can decent work be done.

I've seen similar sample sizes in medical journals, but I wouldn't necessarily call those junk.

From my personal experience, the problem is that people who study psychology are not interested or skilled in statistics. Unskilled professors teaching uninterested students perpetuates the cycle.

The field is slowly improving but still has a long way to go.

> I've seen similar sample sizes in medical journals, but I wouldn't necessarily call those junk.

Unless those small sample sizes are justified by rigorous statistical modeling, showing that the result is still significant even with the small sample size, such studies absolutely are junk.

The burden is on the researcher to demonstrate that their experimental design is suitable for obtaining meaningful results. Failing to do so is the definition of junk science.

And needless to say, lack of statistical knowledge is not an excuse for not doing this. If a medical scientist is unable to perform an accurate statistical analysis of their data, it means that they cannot publish – not that they can skip that part because it's too difficult for them. The entire study is meaningless without it.

Analysis of statistical power is a super basic thing and is a normal part of research in medicine, psychology, social sciences ... etc.

Sample size is just one factor. For example, if you are interested in heights and you measure six people and they are all over 10 feet tall, you can go ahead and conclude they are from a group with different height distribution than the rest of humans.

Pretending that these papers of “wow, we found a substance that when given to ten people in a cancer group with a 99% six month lethality and three survived more than two years” contribute nothing is a weird belief. Penicillin was what n = 1.

Psychology attracts people who are interested in working with people than working with things. If you include the undergrads, psychology has a lower IQ (on average!) than say mathematics or engineering, or other soft sciences like economics or data science.

But you shouldn’t judge a group by its average. Also, it’s misogynistic in this case, obviously.

Psychology isn't accepted as a hard science, perhaps an applied science. Since there really isn't a precise follow to the scientific method, hypothesis - inference - conclusion.

So Astronomy is not a hard science? Because there is only observation, no testing of hypothesis?

They can test it in the limited sense of simulation. Not ideal but it's more than even psychologists can do

That is so completely false that I fear you are responding purely by a bias against psychology and have not read anything about it. Of course you can form a hypothesis, and test it on real live humans, gather data and check results.

I don't think a lot of people know this. The evidence required for a physiological intervention is lower than a surgical or pharmaceutical intervention. The AMA used to lump psychology with things like chiropractic, acupuncture and homeopathy until they were sued by chiropractors under antitrust law in 1987.

I wish there was a succinct way to communicate this. Lots of people I know revere psychologists' word as end-all be all, beyond doubt. To question it makes them think I'm questioning physics.

There are good ideas in the field, to be sure. But on the other hand...

For me the scariest part is its the only speciality that we allow to imprison people (involuntary holds) and they can keep people out of prison (not fit to stand trial). They also often influence public policy. That's a lot of power for people who can't do a double blind study most of the time.

It's even worse than that: Some jurisdictions (including California IIRC) allow involuntary psychiatric holds based on the assessment of people (like law enforcement officers) who have no psychological qualifications whatsoever.

This is without a doubt one of the worst and most normalized human rights violations happening in the world today, and the scale and impact of it is hard to imagine.

I have seen police arrest people under section 136 of the UK's Mental Health Act in circumstances where no expertise is needed to see that the person is a danger to themselves or others.

The police then take that person to a place of safety, usually a hospital.

People seem to be confusing clinical with academic psychology. While the former uses a lot of the more well-established concepts(like conditioning) from the latter, it's very much its own discipline, and it actually has much more rigorous research methods than academic psychology because treatments are subject to regulatory approval.

I can speak from a lot of experience when I say I would trust a clinical psychologist over a psychiatrist any day of the week. Psychiatrists are just doctors who learn to prescribe medication based on a simple set of rules and psychometry. There are psychiatrists who take their field very seriously, but that's the exception rather than the rule.

Here's the thing. There is a decent body of evidence for chiropractic and therapeutic touch if I'm honest. People find relief from pain by going to a chiropractor, but they also find similar results getting a massage.

It's basically impossible to separate the benefits of talk (social interaction, validation, etc) from the talk therapy. Lots of people whould probably be served just as well if they had someone much less trained, or not even trained at all to talk to in a similar manner.

Furthermore as the social bonds disintegrate studies suggest that individuals have less friends and less interpersonal interaction generally (social media use excluded) at the same time the need for psychologists has increased. In some demographics over half the members have been diagnosed with a mental illness. How can we be sure that all of the sudden lots of people have become mentally ill vs lots of people need someone to listen to them?

I generally agree with your point about the benefit of talking. I think there are many cases where someone would probably benefit as much from just talking to a close friend. But there are also many cases when a therapist can provide objectivity, discretion, insight and experience that friends just can't match.

This goes especially for actually severe mental disorder, because the thing there is, those patients have things going on in their head that most people can't relate to or understand(like irrational fears for instance). Psychologists however have a lot of experience talking with these people, reading and interpreting abnormal body language. And they know the sort of things you definitely don't want to say to a patient.

As for your point about social media, my personal view is that you're mostly right except I do think real mental illness is increasing, because loneliness is such a strong cause of depression especially, but also social phobia.

Throughout my childhood most of my interactions with people outside my family consisted or bullying by other kids and neglect by teachers. Because of this, it's deeply instilled in me that in any social interaction people are catalogueing my every mistake or negative attribute, or just ignoring me. No amount of conversation with other people(or medication btw) has helped me overcome this, but the only progress I have made was in therapy. What I really needed was help to understand why my brain works this way, and how I can be more aware of my own irrational thinking. This has helped me to obsess a lot less about social situations. Though I still have a deep mistrust of people at my core, I've at least been able to let some people in and learn to trust them. And accept that they really do like me for who I am.

While a bit rambling, I hope that provides some context on what a good therapist can do compared to speaking to friends and loved ones.

I'm not saying you shouldn't go to one on a personal level. If it helps you or improves your life it doesn't matter if scientific or not, it isn't anyone's business. I'm speaking more about it from a population level. I think we could probably come up with ways to more cheaply (read: increase access) to help people with moderate mental illness (that I suspect in some cases may be the brain working correctly and accurately finding it's environment to be unpleasant and lonely).

For young men without strong male influences in particular I think a kind person in some sort of big brother program is probably worth a team of psychological professionals in many cases.

Sounds like somebody should design a study and test it.

I don't think you can blind it properly. Patients will probably report the care as more effective from therapists due to social conditioning and placebo effect. I recall a study where doctors in white coats were rated more effective than without by patients.

I definitely agree that we can do more to help moderate mental illness, and that a lot of things can be more communal than clinical. That's all great.

But we need to recognise that actually, there are people who can't be helped this way, who need more than just any talk at all, and that clinical psychologists have a lot of useful expertise on how to help these people, that can't simply be written off as unscientific or just a friendly conversation. There's a lot more to it than that.

Reply on news.ycombinator.com