Why Science is Failing Us
wired.com
wired.com
The article basically seems to be relying on some philosophy and muddy and different definitions in different contexts (philosophy vs science vs standard usage) of words like "fact". Then it talks about how things are getting more complex and we're spending more effort to learn things now than we used to because "we know all the easy stuff". And seems to conclude that we'll still be no better than religious shamanistic people once we "know all there is to know" and it still won't do us a lot of good.
It seems to be advocating give up on science now, with some rational like "while we're ahead".
I honestly don't get it. It seems like cloudy wooly thinking, bad arguments.
Sure, things are getting more complex and will continue to, but that doesn't mean we should give up, or that "it's mysteries all the way down". Every year we learn more and fix more problems. And we have to and always have had to make a lot of mistakes in the process. The author seems to think we're making more mistakes now and that's an indication the game is almost up.
I disagree, medical science is still churning out amazing breakthroughs, like HIV and cancer vaccines this year. And physics is still coming up with amazing things.
Just because it's getting harder doesn't mean we should stop or that we'll hit a wall and be able to go no further (and if we can see that wall coming we might want to think about stopping prematurely?)
Everytime we thought we'd learned everything we've been able to push on and learn more, discover more depth, and use it more to our advantage. I don't strictly speaking see why that has to stop just because it's getting harder. At least any time soon. Each new level also gives us better tools to work with.
And there have always been people saying we know enough now, or it's getting harder so lets stop now. And some have, and many haven't and that's why we still have progress. This is a age old endless reoccurring trend and bares the same ignoring it has always gotten. Or you can step off the train of progress and be left behind.
I do not think science is failing us at all in anyway. I think this article is poor on many standards.
Making the article relevant to this point, Pfizer spent tons of money developing a drug and bet it would work simply because (at the time) lowering HDL = better cholesterol. No one asked what does HDL do in the body? What does LDL do in the body? What depends on cholesterol?
It's kind of like when you come up with a really cool idea but forget to look at the target market to see if it's actually viable.
The bigger point is that all of these thing go throug proper testing (double blind, phase 3 etc) and this bigger system does catch them before we just call them facts and unleash them. We are testing them and figuring it out, it's just complex and hard.
Same with the search for the Higs Bosen. We're pretty sure it's there, it seems like we even have a bead on it now, but we're still searching and testing big ranges and it takes time. I'm not sure you can just come up with a new way of reasoning to find it.
But of course you are welcome to try, just don't tell everyone else they are wasting their time while you are trying, because you might be wrong and wasting your time.
The article did leave something critical out: what now? Reductionism doesn't seem to be working for systems like the human body (or psychology or climatology or...). So what do we do about it? I think that is what my grandparent reply meant by a new process to deal with complexity, because reductionism is failing to often in these systems.
Sure newton was essentially wrong, but a) newtonian physics is still a fine short hand for near earth simple stuff, and b) it's not like we lost understanding of the world, it was supplanted by a better more precise understanding
In theory, yes, in practice, no. The human body is wildly complicated with an absolutely enormous state space, surrounded by an incomprehensibly larger possible state space that expresses our ignorance about what the body might be actually doing. Throwing a dart at random and eliminating some particular claim about the nature of that state space may impart vanishingly small fractions of a bit of information, roundable to zero with insignificant loss. Not entirely without value? Sure! Worth it? Well, losing the bet and coming up effectively empty in information a certain amount of the time is inevitable, but when you pretty much do nothing but lose, it's time to change your strategy.
But the author said "Science is failing us" not "Biochemistry".
And I still disagree, there is still tons of new amazing and effective stuff being churned out. If you want to draw a line in the sand and try not taking and new medicines invented after today and see how you do fine, but I'm going to keep up because I still think there is a lot that can be done and we've barely started
Point well taken. Human arrogance does always seem to be the largest problem in any system.
Human biology is an extremely complex, chaotic system. Just because we don't know everything about it yet, that doesn't mean we will never understand it, or that we are not making progress.
You have a good point about finding ways to look at multiple causes and effects at the same time. There is some very cool work using machine learning techniques like clustering and SVMs on transcriptomic/proteomic data to try to understand relationships between genes. I would be very surprised if they don't have people doing this at Pfizer, but maybe having a few more could have helped them in this case.
On a work-a-day level, scientists and engineers seem to take the same position on causality as that proposed by the medieval scholastics.
For instance, the author chose to talk about Hume but chose not to mention Popper. Falsifiability is the key word here. Science doesn't advance by gathering knowledge about how the world works directly, but by accumulating false results.
And besides, the fact that there will always be mysteries is what makes science fun in the first place.
I read it as no one knows everything, always proceed with caution.
If you're on the bleeding edge of particle physics then sure, things are getting more complex. But most discoveries to be made in science are relatively mundane everyday things, and just a matter of paying attention in a structured manner.
E.g. most forms of life on the planet haven't been studied in any detail for the simple reason that nobody's gotten around to it yet, but the methods and tools to do so are much easier than they were 50 or 25 years ago.
Anyone can contribute to science, and it's now easier than ever before, and some of those contributions might unearth something we didn't expect.
What sort of study could a lone wanabe do that would actually be publishable?
A huge fraction of sea-life is "undiscovered", especially small stuff. Yes, even at scuba depths.
He's saying not that the method is bad, but that things are getting complex enough, at least in medicine, where we're hitting some limits on how easily we can practice science to the necessary level of rigor and precision.
But we are still doing fine and achieving the "necessary level of rigor and precision". The one main example of the article was the drug that was stopped in trials, which is exactly as the process was supposed to work, and yet then the author tries to paint this as a reason or example of the failure of science in general.
Not that this is a particularly bad approach. It worked amazingly well for the last few decades and has churned out a lot of blockbuster drugs.
It is, however, starting to fall apart as all the low hanging fruit has been picked. It's time for the big pharma's to start doing science again. The problem is that big pharma is thoroughly infatuated with quarterly stock prices and not developing new drugs. They are cannibalizing themselves in an effort to keep stock prices up and profits growing.
It's better to think of big pharma as remarkably successful marketing engines, not medical science companies.
For instance, I know of many scientists that work for big pharma companies which do not have access to any of the scientific literature (past the freely available abstracts). Management will not allow them to purchase these articles.
Consider that for a moment. The companies producing your drugs are not even remotely up to speed on cutting edge biological knowledge.
*Obligatory "I'm in biology so I'm not making this stuff up" disclaimer.
More than that, the pharmas are absolutely resistant to embracing technology that can save them; they discard scientists that are driving their respective fields forward, and resist the notion that understanding the cell as an information system can provide better returns on their trials.
There are exceptions, when Genentech was run by Art Levinson (a scientist) he was capable of discerning science from bullshit and was an effective CEO. However, I would sell any stock in a pharmaceutical not run by a scientist; such a company may be able to post short-term returns but they're only doing that by selling off any possibility of future success.
Jeffreys, Diarmuid (August 11, 2005). Aspirin: The Remarkable Story of a Wonder Drug. Bloomsbury USA. p. 73. ISBN 1582346003.
But there is an idea in the article, which does seem to be true: most of our current problems are industrial-technology related (environment, energy etc. consumption, known as "externalities" in economic lies, which is basically abusing Earth -- and therefore ourselves -- in the current "economic" system) and just doing more of the same might not be the way out.
Another "critique": All watched over by machines loving grace and Henrietta Lax by Adam Curtis (BBC documentaries). Adam goes further, with TINA.
I think both you and the author are incorrect. Medical research has basically become an elaborate scam for not just the reasons mentioned in this article, but at least a dozen other reasons as well. If you look at the last fifty years of pharmacological/nutrition research, there really haven't been more than a handful of important discoveries despite the literally trillions of dollars that have been spent.
Notice how new drugs are given the least emphasis of the four factors listed. Plus, one of the biggest reasons so many people are getting cancer is the flawed nutritional advice of the medical establishment.
Anyway, clearly there has been some progress, but we would be in a whole other place if the vast majority of the research dollars hadn't been completely wasted. Not to mention all the potential cancer cures that have been made illegal, meaning that it's not even clear the impact of the new drugs is making up for how the medical establishment is screwing us all to begin with.
2) Regardless, I think your expectations are warping your perception.
Because the article was about pharmacology and I said pharmacology in my post.
"Medical research has basically become an elaborate scam"
http://en.wikipedia.org/wiki/History_of_cancer_chemotherapy#...
They are probably hoping for responses such as yours. It's engineered magazine marketing, and not actual content.
I'm not a scientists or anything close to that, but I agree to most of what you sad. What I wanted to add is that there's something extra to be sad about the "let's stop now" part.
I'm talking about things like human cloning and, I don't know what its exact name is, "organ harvesting from human clones", let's name it that way. I know that for the most part this is still SF-material, but we're getting there with very big steps, and I think a serious discussion on sensitive subjects like these which would also involve the scientific community must take place rather sooner than later.
And before anyone accuses me of being a "religious bigot" or something like that I must tell them that I don't care at all about religion, but I do feel that scientists themselves should stop burying their heads in the "we only do science here"-sandy dunes, and confront the real world.
Correct me if I'm wrong, but I get the impression that he thinks our current approach to science is wrong due to our tendency to attribute cause and effect to things where we have no conclusive proof of cause and effect?
Well, if he has a better suggestion on how to approach research, I'm all ears!
First off, I don't think ANYONE who was involved with the development of torcetrapib thought it was a "slam dunk". The success rate of drugs that have reached phase III is only slightly north of 50%.
Second, there is no way you could possibly figure out all the effects a particular drug has on the human body. You'd be doing research for the next 100 years and you still wouldn't come close. So what we do is we come up with a hypothesis (high HDL is good), we gather evidence in the most efficient manner we can (other drugs that raise HDL help prolong life in humans and animals), then we move forward with our BEST GUESS. That's how science works, you create a hypothesis, then test it.
Are our hypotheses wrong sometimes? Of course. Do we learn something from the failures? Yes. Trying a being successful 10% of the time is far better than not trying at all and being successful 0% of the time.
Merck's CETP inhibitor is in phase III right now and there is a chance that it will fail too. And I don't think any scientists feels that high HDL is the cause of reduced cardiac risks. A more accurate description would be to say "High HDL is associated with reduced cardiac risks, this drug increases HDL levels, so it stands a chance of reducing cardiac risks".
I think the author does a bad job of describing how scientists approach their work. If anything a scientist would be the first to call out a claim that something _causes_ something else. That's how their trained!
I think there needs to be revision of the experimental method overall, as the models of drug development in the 20th century simply aren't sustainable or efficient for the diseases of the 21st century.
As a culture I think we have a certain faith in the medical system to save us from death, the same way previous generations looked to priests to save the spirit.
The inference you mention 'high HDL is associated with low cardiac arrest, therefore a drug that increases HDL might help is more vague than a layman expects. It's like saying 'I want a safe car, and German tend to be safe, therefore I will buy a German' - it's valid in the absence of a real understanding of how to specify and select a safe car, but it's more vague than you'd be comfortable with. You expect an engineering company to be able to specify a safe car based on deep knowledge. But because our understanding of the disease, and of what different chemicals can do is incomplete, a drug company can't do that. Instead, they follow as many hints as they can to select a chemical that might work, and then advance it through a series of progressively more expensive trials until they are pretty sure it does more good than harm.
That's a valid way of doing things, and at the moment it's all we can do. But it's not what a layman imagines, or certainly not what this one imagined. It undermines our sense of control - our sense that we are immortal and can get on with making an angry birds clone to get rich because there will be plenty of time to do the projects we want to after the payoff - it's not like we are going to die of heart disease, science has our back on that one!
Or maybe I'm generalizing my personal feelings too much?
However, there are plenty of conditions you can develop where modern medicine is only able to contain the symptoms, rather than fix the problem causing them. Autoimmune conditions (such as MS) come to mind here.
The main point of the article is that scientists have exhausted the low-hanging fruit of useful correlations and are now grasping at the more dubious correlations. The author claims that things are complicated by the concept of causation.
He cites David Hume: "...causes are a strange kind of knowledge. This was first pointed out by David Hume, the 18th-century Scottish philosopher. Hume realized that, although people talk about causes as if they are real facts—tangible things that can be discovered -- they’re actually not at all factual. Instead, Hume said, every cause is just a slippery story, a catchy conjecture, a 'lively conception produced by habit.' When an apple falls from a tree, the cause is obvious: gravity. Hume’s skeptical insight was that we don’t see gravity -- we see only an object tugged toward the earth. We look at X and then at Y, and invent a story about what happened in between. We can measure facts, but a cause is not a fact -- it’s a fiction that helps us make sense of facts."
It's been a while since I've taken philosophy, but Hume's skepticism of causality is itself a story by its own criteria.
Sure it does! Extend the analogy a bit by letting the tree be a living thing. That today you picked some fruit doesn't mean that tomorrow a new pear (apple, guava, grapefruit, etc.) won't grow where you harvested.
And when the tree dies, there's always that "making lemonade from lemons" metaphor... (OK, so that's engineering, not science, but it will keep you busy and productive).
A lot of people want science to be like politics. They want you to pick a side and stick to it regardless of the data.
IMO, when conventional wisdom isn't at least occassionally overturneed -- that's when I'll begin to think science is failing us.
Fix the incentives, and I think we'll have better science.
Not sure how you would 'fix' those research companies.
Any by pre-registering, you can follow up on research that has been silently "forgotten about". If too much research is forgotten about, you stop trusting the results you do hear about.
But: What if they instead do research secretly. Then on research success, they publicly notify the journal of what they intend to do and that it will take 5 months to research. Then 6 months later (an extra month to look plausible), they publish their successful data.
If science was really failing us, I don't see how we would have managed to retract those incorrect studies. It feels like the writer had no bigger point than "biomedical research is hard, let's go shopping." It's sensationalist to consider science a failure every time it makes a mistake.
I thought the Hume references were pretty bad, too. If you read what he says, he questions the existence of relations such as "A causes B" and prefers to phrase them as, "In the past, we have observed A-like events are always correlated with B-like events." For practical purposes, that's enough to behave as if causality "really" exists. You just have to avoid mixing up causality with mere correlation, which every good scientist already knows.
Claims like this one:
> "First, all of the easy causes have been found, which means that scientists are now forced to search for ever-subtler correlations, mining that mountain of facts for the tiniest of associations."
May be true, but the author presents no evidence to support them, with relevant studies, articles, etc..
The second, much more troubling problem however, is that the argument suffers fromt the very problem it presents! The author's conclusion that the returns on scientific research are diminishing due to an inherent flaw in conclusions being drawn from correlations-- is itself a correlation. He's correlating the increasing cost of research to the increasing difficulty of finding new correlations in the data.
There are other, simpler, less circular and philosophical explanations for why the returns on pharmaceutical research have decreased, such as increasingly strict regulations and fear of risk on the part of regulatory organizations. See this TED talk, where Juan Enriquez talks about these issues: http://www.tedmed.com/videos-info?name=Juan_Enriquez_at_TEDM...
All of the examples are pulled from medicine, notorious for its lack of experimental rigor. To say that "Science has failed us" implies that either medicine is the the only important science or that all science is equally as sloppy, which is pretty insulting to a scientist in any harder field.
His focus on causation is even more misguided. The very purpose of science is to understand the way the world works; to understand what causes what. To attack the idea of causation is to attack the very idea of science, and in turn all of the advances it's brought about over the past 300 years. Beyond that, we implicitly accept causation in almost every aspect of our lives (Pressing the space bar causes a space to appear, etc). Certainly causes can't be 'seen' like facts, but to suggest that this trivializes them, or somehow makes them less useful is nonsense (and, for what it's worth, is total misreading of Hume).
Complexity isn't a valid reason either. Some very well understood systems are incredibly complex (look at the computer you are using now). What is true is that like all other humans, scientists make mistakes. We often make the incorrect causal links or are influenced by our biases. This is why experiments exist (instead of pure data collection); to make sure the causes we have assumed are correct. To point to a couple of experiments with an unexpected result and then say that all of science has failed isn't even a little bit right.
If you were to say that medical research _accepts_ a lack of experimental rigor (simply because there is no other option), I would agree with you.
That said, I think he is correct in his critique of medicine/pharma. The cost of drug development has gone astronomically high these past decades, with billion dollar pipelines to account for the cost of failure.
The pharma drug development model has not really evolved beyond a lottery system of testing random compounds to treat diseases, and going back/forth until the right permutation of a compound is found. This might have worked before for initial "easy" diseases (that had easy drug targets, or single gene mutations), but the problems we face now (Alzheimers, Cancer) are too complex for our lottery-based drug development system.
What I think has really changed is the cost of failure. The best example I can think of is the discovery of benzodiazepines (the drug class that includes Valium). The first benzodiazepine (chlordiazepoxide, Librium) was discovered in 1957 (we're talking, the FIRST set of pre-clinical tests) and it was on the market in 1960. 3 years from the first tests to market.
Nowadays, you'd be lucky to get to market in 15 years. A great example is Qutenza. The product is nothing more than a patch that contains a very high level of capsaicin (the stuff that makes peppers hot). When you apply it to the skin, it can reduce the pain that sticks around after an attack of shingles. I can't think of a product with fewer safety issues, yet it took 10 YEARS for the company to get FDA approval.
This is due to a combination of increased FDA scrutiny around safety along with a high standard for efficacy (i.e. we don't care if your drug reduces cholesterol, we want you to prove it reduces heart attacks). So in the past, when a smaller, shorter trial was sufficient for FDA approval, you could take a promising drug all the way to the FDA without a lot of expense. Not so anymore.
Science has never failed us, as science is inherently just human curiosity. The continuing structural growth and big bureaucratic developments that many governments, schools, and businesses are implementing are failing us.
"Bureaucracy is the art of making the possible impossible." - Javier Pascual
Science can't fail. It only illuminates. But a lot of shady assholes run this place, and the last thing they want is a light shone on them.
I think he has a basic misunderstanding of science. He doesn't realize that "made up stories" to explain how things work are just a starting point to understanding. They have to be tested and revised and re-tested until we come up with an explanation that reliably predicts how something works. And failures are an integral part of the process. Failures advance understanding.
It reminds me of the quote by Thomas Edison, after a thousand failed experiments, "We now know a thousand ways not to build a light bulb".
Take for example the first time you came up with a cool new product. You took it to a VC/someone who's done it before, and they ask you about your market, price, revenue etc. Science is still creating cool new products, not paying attention to everything else around it.
Here's another good example: wind farms. We've been creating massive new wind mills that are more efficient bigger, etc etc. Have we ever looked at how to install them in such a fashion that they become more efficient as a team rather than an individual? And have we looked at how wind patterns change because of them?
I am curious to see if the author has any actual suggestions on how to do science.
A more interesting limit is relationships that cannot be understood in isolation. When these exceed our working memory, we can't perform our usual trick of hierarchical abstraction to look at one part or one aspect at a time. Perhaps that could be our limit of intuitive understanding, unless we come up with a fundamentally new way of understanding complexity.
http://norvig.com/experiment-design.html
and if we learn the lessons of how to interpret research findings more cautiously, we can all do our part to guide further research better.
As the author of the submitted article points out, "This doesn't mean that nothing can be known or that every causal story is equally problematic. Some explanations clearly work better than others, which is why, thanks largely to improvements in public health, the average lifespan in the developed world continues to increase. (According to the Centers for Disease Control and Prevention, things like clean water and improved sanitation—and not necessarily advances in medical technology—accounted for at least 25 of the more than 30 years added to the lifespan of Americans during the 20th century.) Although our reliance on statistical correlations has strict constraints—which limit modern research—those correlations have still managed to identify many essential risk factors, such as smoking and bad diets."
So with caution about assuming causation where the data cannot reliably show causation,
http://escholarship.org/uc/item/6hb3k0nz
the huge task of biomedical research can still go forward, eventually yielding other findings that can improve health or longevity compared to today's baseline.
AFTER EDIT: The question posed in the first reply below is interesting. One reason that biomarker interventions are tried more often than "hard endpoint" interventions is simply that they are faster and easier. To really check carefully for hard endpoints--reduced mortality and morbidity, for a medical treatment--takes time in a clinical trial. Sometimes an effective on a biomarker, for example serum cholesterol, can be observed right away, but if the subjects in a study are at an age at which few subjects die from any cause, it can be a long while before a study reveals which treatments actually increase rather than decrease the risk of death.
The case of the drug rimonabant,
http://en.wikipedia.org/wiki/Rimonabant
which had reasonably strong support from animal experiments as an antiobesity drug, is instructive. Studies of human subjects after the drug was approved in Europe revealed a huge increase in suicidal risk among patients taking rimonabant,
http://www.pharmacist.com/AM/Template.cfm?Section=Pharmacy_N...
and eventually approval of the drug in Europe was withdrawn, and the drug was withdrawn from the market by its manufacturer, before rimonabant was ever approved in the United States.
What happens is doctors read the correlation and start massively prescribing biomarker targeted remedies (such as vitamin B) before any causation is shown. Why does this happen?
I'd guess it's because promoting this possible causal link as a cure for diseases is pretty much the easiest way to get it tested---where else will you find 25k volunteers?
As always, when people are involved, the reason for failure seems to be in the incentive structure.
How can this possibly be true? Completely incapacitated, ie bedridden and immobile? Surely the country would be in near collapse if 30+ million of its inhabitants were randomly bedridden at any one time just by that one medical issue.
It is an overstatement. But if you merely ask around, especially of men, you will likely find a majority of them have had serious back pain and have some trepidation that it might, at any time, for little or no reason, return.
I've got to try to get my socks on, now.
Way too long, meandering, full of anecdotes.
How is it at all surprising that trying to "fix" problems with the body is uber-complicated?
Plus, there's the obvious missing bigger point - all these companies are trying to find a solution that is a pill, as opposed to changing the underlying problem: bad food, bad environment, bad physical conditioning, etc. Billions spent on finding pills, very little money in solving the root causes...