Hacker News Reader

Comment by TeMPOraL | Hacker News Reader

Judging by the replication crisis discussions over past two years, I'd add quite a bit of medicine to that list too. And I don't mean just psychology, which is social science with a medical degree - but also drug research.

All of this, though, shows how hard those disciplines are. Relative to that, hard sciences are easy, because they study precisely defined things pretty closely entangled with laws of physics, and you can't easily bullshit mother nature. But as you move away from physics, complexity rapidly increases, the questions themselves become fuzzier, and the answers less clear.

So IMO, the probability that academic success is crapshoot or scam increases with (Kolmogorov) complexity of the subject domain.

View on HN

In the case of at least medicine, there is another degree of hardness involved: sample size. While in physics, for example, sample size is limited by cost, space or, in few cases, frequency of occurrence, in medicine, a lot of studies are limited in size because of ethics. Many studies, especially drug trials, have an risk for at least one group. It is unethical to put more people at risk then absolutely needed, so studies are normally designed so that signal-to-noise is the minimal viable.

not jusT ethics. Cost is a huge factor as well for clinical trials.

Cost is a factor for sample size in any field: but physicists don't publish if they can't accumulate a sufficient sample. As a result physics experiments where sufficient sample sizes can't be obtained are, simply, never performed - and as a result physics as a field grows in the direction of high-certianty.

Physicsts absolutely could be performing experiments with poorer statistics; it'd be cheaper, more advanced things would be done sooner, and we'd be having a replication crisis.

If the social sciences were willing to go a century without discovering anything exciting, they could achieve whatever level of certianty they wanted on their current budget. (At the expense of new discoveries.)

High certainty but that comes with second order issues: many ideas are untested, and the ideas that are turn into papers with 100s of authors. This stifles independent thinking.

Here's a modern experimental paper. Scroll down to the authors. You'll know it when you see it.

https://cds.cern.ch/record/2282544/files/arXiv:1709.05260.pd...

Your statement about huge numbers of authors (as well as the great number of theoretical papers that have to wait a very long time before being confirmed or denied) is true.

Wow the word-per-author ratio is close to 1. What's behind this padding?

It's not quite that medicine is more complex, it's that Medicine is practical and accepts not understanding the details. If drug X works then that's enough we will use it even if we don't understand exactly why it works.

The downside is we end up building simplified models well Y is correlated with bad things so we should treat Y as a bad thing. Often this actually works really well, but also opens the floodgates for all sorts of issues.

That's not to say a less practical approach that simply looked for the details would be more effective. But, it would look very different.

> It's not quite that medicine is more complex, it's that Medicine is practical and accepts not understanding the details. If drug X works then that's enough we will use it even if we don't understand exactly why it works.

I'm always baffled by this. I can understand why we should move these drugs into practice. But, if the way you conduct research stops at X works - you should have a good look at the way you conduct research. I can completely understand short term gains but ignoring long term gains is very unscientific and immature.

I'm afflicted by three chronic conditions which have no known causes. All of them are treatable; there are guesses as to why the treatments work, there isn't really a good way to establish a sound reason for them working.

You can't just take a person apart to study them while they're alive, and you can't easily reconfigure them to account for variables either.

They'd love to understand why X works, but a whole lot of people could be treated by X while they spend all that time and money figuring it out.

Besides, there's plenty of Y where they were sure it was going to work, it fit all the models and theories, it did the right thing in mice, and then in a trial Y fails catastrophically.

> It's not quite that medicine is more complex, it's that Medicine is practical and accepts not understanding the details.

no, medicine really is that much more complex (which is not to say physics isn't complex and that physicists aren't very intelligent). a model of a single-celled organism, or even of a protein structure, is orders of magnitude more complex than, say, the three-body problem or the navier-stokes equations.

we're at the very beginning of understanding truly complex systems situated in truly complex environments, which is what medicine faces. we have millenia of exciting medical discoveries in front of us.

Physics is unbounded; it has infinite complexity at the extreme. Medicine is vastly more complex than say CPU design, but it still operates on well understood chemistry.

And in XXX century it might flip the difficulty of the disciplines around. As it is right now, physics is dealing with relatively simple systems compared to medicine, which deals with absurdly complex systems.

Since you bring up CPU design, the appropriate comparison would be that medicine is to chemistry what CPU design is to physics, if by "CPU design" we mean reverse-engineering alien technology that operates on the edge of our understanding of physics and way beyond our ability to manipulate the physical world.

Medicine is practical and accepts not understanding the details because it has no other choice. In that regard, it's similar to social studies because the latter can't get into details without fully reverse-engineering a human brain either.

> Medicine is vastly more complex than say CPU design, but it still operates on well understood chemistry.

I'm not sure a researcher enjoys the same freedom to put together medical experiments as physicists and chemists do.

Moreover you comment sounds very disingenuous. It's like claiming that electronics is a realtively simple research domain because it still operates on well understood electromagnetics.

Meanwhile do keep in perspective that even today's state of the art in medicine involves various forms of mutilation.

Medicine can do just about anything with small clusters of cells and nobody would complain. That's not everything, but it is quite a bit.

Consider we generally do blood not lymph tests. That's a mix of practicality and tradition, but one one of many areas of basic research mostly left alone. Sure, doctors physically examine for enlarged lymph nodes, but that's because the immune system is largely based around the lymphatic system because that's mostly how diseases move around the body not through the blood.

Getting back to what I said, we can do a lot of high quality medical research that may or may not be useful in the long term, but we don't approach it as a research problem we approach it as an engineering problem looking for solutions to real world problems.

PS: As to CPU vs Medicine vs Physics. We are comparing numbers do large it's hardly meaningful, 10^20 (CPU) vs 10^30 (Medicine) vs infinity (Physics) all round to bigger than you can comprehend.

> And I don't mean just psychology, which is social science with a medical degree

Psychologists don't have a medical degree, psychiatrists do and they could hardly be called social scientists.

Hard sciences also don't have a very high replication rate, they have the same incentives which the article talks about, and the same publishing bias problem.

Part of the replication issue is that even if the relationship exists, it's quite likely a small study will find nothing.

Reply on news.ycombinator.com