Science has lost its way, at a big cost to humanity
latimes.com
latimes.com
The institution of science is undergoing a catastrophic decline. The reason behind this is simple: it is no longer a growing economy. Public funding for science is frozen or being cut, private R&D labs are shuttering their doors, and companies are increasingly concerned with quarterly results at the expense of long term research.
And why should it be otherwise? Science has never payed off as a logical financial investment. It is the riskiest of gambles by definition, requiring inordinate expenditures of time and resources in the present for a chance at some distant breakthrough decades or even centuries in the future. Institutional science is not an economically sound choice in the best of times, let alone during the current span of never-ending recessions.
The truth is, science is a creative pursuit much like the arts. Like the creation of literary masterpieces or profound paintings, it has never made economic sense in the present. Only afterwards, once the impact can be seen, do we understand its significance. And that is why it will always be worth pursuing.
The reality is that, increasingly, we live in a society that does not understand this philosophy of life. People only care about how they will survive tomorrow, and who can blame them, as the world economy gets ever more competitive and cut-throat.
Increasingly, it has become clear that our society does not reflect one designed with its own best interests at heart. Why this is, how it happened, and how we can change it, will be the greatest challenge of our lifetimes.
The problem is that this article generalizes "science" to the life sciences. Biology and medical experiments take a long time, are very expensive and reproducing results can be very difficult. Of course there's more pressure for scientists to produce results and you'll see fishier papers to justify their big grants.
Research can be incredibly profitable. What about Bell Labs? 3M? The technology from your iPhone wouldn't have been possible without Apple's big R&D department. There are many examples of successful (and profitable) R&D departments. Heck, Exxon's R&D arm comes out with many innovations a year to make pumping oil out of the ground easier. It just isn't as visible (or linkbait-y) as that new gene they found.
>> "[science] has never made economic sense in the present"
Life sciences don't. It takes decades for new drugs to come out sometimes. So yes, there are a limited number of people who do this. But electronics? Tech? Mechanical/Aerospace engineering? SpaceX, Intel and others are constantly improving.
>> "Increasingly, it has become clear that our society does not reflect one designed with its own best interests at heart. Why this is, how it happened, and how we can change it, will be the greatest challenge of our lifetimes."
Bullshit. There's financial incentive to go with the fold and do conventional things, but there's also a lot of financial incentive to innovate. However, innovation is hard when there's a higher barrier of entry. Information is the industry that has seen the most innovation in the last decade because anyone and their mother can make software.
The real problem is making the barrier of entry lower for other industries. To make manufacturing, biomedical research and others easier. And guess what? There's economic incentive to do that, too! People who make software development easier are getting tons of money and 3D printing, which will completely change the way manufacturing is done, will probably create some billionaires in the next decade.
The system is fine! In the last decade, we got portable access to all of human information, the ability to always know where we are, advancements in electronics, cars and all sorts of myriad incredible things. Give us another couple decades and we'll make things that will blow anyone from the present's mind.
I can't believe the old "science isn't profitable" canard is being trotted out to full effect on HN. The fact of the matter is that science is imminently profitable, and this is so obviously true that it takes nothing short of a true believer in the benevolence of the government as the superior investment entity and the free market as bogeyman to believe otherwise.
I mean this isn't /r/politics, is it?
The desire for profit is the single largest driver of scientific and technological innovation, period. Yes, NASA and DARPA have created some amazing things, but what about the hundreds of billions which are spent by privately held entities' R&D departments every year? Does all of that research yield nothing of value? The pharmaceutical advances? The electronic advances? The advances in automobiles, computers, housing, information access, and, now, space travel, as well as virtually every other product on the planet - do those mean nothing?
Of course they do. Private funding for research constitutes the lions share of research funding, and that private research leads to the vast majority of scientific and technological advances around the world.
The line that science would not happen absent government funding is so demonstrably false that to even make such a claim is tantamount to shouting to the world that you are a first-class idiot.
This isn't reason.com, is it?
> The line that science would not happen absent government funding is so demonstrably false that to even make such a claim is tantamount to shouting to the world that you are a first-class idiot.
The private sector doesn't advance science. Also: you are stupid for claiming so. Maybe even first-class stupid.
See where that exchange gets us? Now cite your data on the majority of scientific advances being privately funded, since you claim to be able to demonstrate it. In particular, I'm interested in basic-science advances, of the kind that CERN and Stephen Hawking and etc. produce. Who in the private sector is producing those?
You don't think that private companies regularly form hypotheses, test those hypothesis, and draw conclusions from their results? Applied science is no less worthy of the title of science than is pure science, and the private sector is applied science writ large.
As for pure science and government funding, I'm not against it, and promoting one over the other is not my intention, as they both have their roles to play. I agree that pure science would not occur to the extent that it does now absent government funding. What I disagree with is that science would not happen at all were that the case. That's flatly stupid thing to say, and I'm a little amused that you would make that claim just before calling me stupid.
In a world where the rate of technological advance is larger than it has been at any other time in the history of the world (and is still increasing), it seems obvious that both are playing their roles rather well. The system is functioning. Science is not dead. Hell, it's not even sick.
Second, Intel advances very specific, safe-bet science like shrinking the semiconductor. As plenty of other people have noted, the semiconductor was a public-funded invention and farmed off to Japan before it could be made "profitable" by American corporations. Heck, early American business laughed transistors off the continent before the value proposition was discovered.
Also, as long as I'm commenting here, I'll note that science is not dead, but I believe that our ability to consume information has tainted our ability to understand true science. To me, the quintessential American scientist of the 20th century was Richard Feynman. His Cargo Cult Science [1] essay, which pops up on HN from time to time explains what's happening best: people who claim to follow science, are actually practicing some sort of modern-day witch doctor magic. They wave their hands and claim the data supports their wild discovery and someone bites. Suddenly it takes 10,000 hours to become a master of something "on average."
Maybe this is just phrased poorly. In that essay Feynman is saying some people (namely advocates of pseudoscience) are invoking "science" to give credence to their claims but are abandoning scientific rigor, this is what he calls cargo cult science. Not all people who claim to follow science.
Seriously? The greater issue here is modern day "scientists" or people who are interested in and consider themselves scientifically-minded, reading something like that and filing it away as new knowledge and not questioning the veracity of things.
As the other child to this comment noted, the problem that Feynman outlines is that no one is doing these tests over again to re-test hypothesis under new conditions or with new devices. The results get published and suddenly it's taken as Truth with a capital T.
"Because there are no facts, there is no truth Just a data to be manipulated I can get any result you like" -- Don Henley [2]
If that makes you uncomfortable, I suggest you try out some form of theism. Because if tomorrow I fall through my floor when getting out of bed, all we can do is re-evaluate our knowledge based on new data. And there's probably going to be a lot of new data.
1: http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.... 2: http://ntl.matrix.com.br/pfilho/html/lyrics/g/garden_of_alla...
"We never are definitely right, we can only be sure we are wrong."
In this fashion, we never hold objective truth in the Platonic sense, rather we accumulate knowledge that explains how the world works.
Does fellatio in fruit flies improve sexual reproduction? One study suggests it does. The problem we face now is that a substantial portion of people, including many very well educated folks who would claim to live scientifically-based lives, say "That's amazing!" and run and tell their friends this hilarious new "fact" they've learned.
Unfortunately there are plenty of these new "facts" people learn that are much more than novelty and lead to public policy or destructive dietary changes cough fructose cough.
We could all use to be a little more skeptical, pragmatic and curious in our lives.
On the other hand scientific research is an example of a positive externality. According to economic theory the market underproduces this kind of goods. There are numerous measures to correct this including patents and public funding of science. We shouldn't dismiss the private companies nor public funding as both parts have positive contributions to science and are necessary for the science to operate effectively.
http://en.wikipedia.org/wiki/Peter_D._Mitchell
He and his former research colleague, Jennifer Moyle founded a charitable company, known as Glynn Research Ltd., to promote fundamental biological research at Glynn House and they embarked on a programme of research on chemiosmotic reactions and reaction systems. In 1978 he won the Nobel Prize in Chemistry "for his contribution to the understanding of biological energy transfer through the formulation of the chemiosmotic theory."
So, yes, the private sector does advance science, even basic science with no profit value. To claim otherwise is first-class stupid and also ignorance of history.
Josiah Gibbs, Irving Langmuir, and Robert Millikan were all American scientists who ran their research off of largely private (some for-profit as in the case of Langmuir, who did both basic and applied science under the auspices of GE; the others at predominantly non-profit educational institutes) These people are not insignificant - Gibbs rewrote the book on thermodynamics; Langmuir figured out that argon should fill lightbulbs -applied- but in basic science also invented the concept of plasma voltage, made observations describing ocean circulation, and conceived of molecular monolayers; Millikan discovered the charge of an electron). I am sure I am missing other American scientists who did their research with little or no help from public sources.
If you want another, more contemporary example, oh hey there's an entire research institute that doesn't take much (if any) in the way of federal funds: http://en.wikipedia.org/wiki/Janelia_Farm_Research_Campus
http://www.janelia.org/professional-opportunities/research-p...
"All laboratories are internally funded without extramural grants."
You probably mean, 'for-profit private sector'. But then, you disregard the several Nobel prizes that Bell Labs received, the intangible impact of IBM's Deep Blue and other programs on computing, etc. If solving a basic-science problem can result in several million dollars of revenue, you can be sure that a company will throw money and man-power at it. There are several examples even among the non-Bell Labs, non-IBM, non-Xerox companies: Google's search algorithms, pharma companies' Lipitor, Viagra, etc., Intel and AMD's industry-defining minuaturizations, 3M's optical films, etc. etc.
Please reveal your source for this belief.
>>every other product on the planet - do those mean nothing?
Breakthroughs are expensive and do not conform to our short term business cycles, as they rarely result in products, as opposed to breakthroughs. Trivial and immaterial incrementalism perfectly conform to our business cycle.
It's interesting that you point to Apple. Compared to companies with a more old-fashioned approach to R&D (Xerox, Bell Labs, IBM, Microsoft), Apple spends relatively little on R&D, and does practically nothing of genuine scientific value. I would hold up Apple as an example of the profitable "focus research on specific products, buy research expertise rather than cultivate it in the firm, forget science" approach to R&D.
Its like if Mercedes Benz claimed "only we could come up with the idea of a steering wheel, seat belts and a motor" when announcing their new model vehicles every year.
Xerox had about 10 years of glorious moon-shot type research. IBM is notoriously fickle about spending on R&D.
The only exception is Bell Labs, which succeeded due to a combination of a number of factors [2]. There wasn't any grand vision underlying it, just the interest to grab the best and the brightest, during a time when there was far less silo-ing and over-specialization than today.
[1] http://www.booz.com/global/home/what-we-think/global-innovat...
[2] http://www.nytimes.com/2012/04/08/books/review/the-idea-fact...
IBM has an excellent track record of scientific contributions, has an excellent research institute, and I gather is good about things like giving its employees time to write scientific articles and providing support to employees who wish to pursue PhDs. Xerox PARC still exists and still does good, publically visible research. I don't see what "grand vision" has to do with anything.
Also, regarding 3M, I am not sure what you mean by 'far lower margin industry'. 3M is a conglomerate. It's like GE, in some sense- doesn't play in any single industry, but in several tens of industries.
R&D Turnover R&D/turnover
Apple 3.4 156 2.2%
Samsung 10.4 188 5.5%Science has enabled us to separate fact from fiction, and hence develop theories that allow us to manipulate our environment in sophisticated ways.
I don't see how you can argue that 'science' as a whole hasn't paid off as a 'logical financial investment'. That claim is so challenging to my worldview, its hard for me to engage with.
For a start, what might we have invested in instead that might have had a higher rate of return?
You couldn't directly build a product on Einstein's major results, for example, certainly not within the time window of patent protection (if they were even hypothetically patentable), but they were hugely important anyway. Same with much of the mid-20th-century materials-science work that is now proving useful to chip manufacturers: the tech industry didn't fund that research in the 1950s-70s, because they didn't know at the time which physics results would be useful to them in 2010s chip engineering. But they're definitely using them now! It's important to the tech industry that this pipeline of not-sure-where-this-will-be-applied basic research exists, because it's hard to do things like improving manufacturing processes unless there is an existing understanding of how physics and materials work in the first place. But the case for them funding it directly is weaker, because it tends to have the property that advancing the general level of knowledge benefits everyone, not only you, and you don't know decades in advance which specific knowledge you'll eventually need, anyway.
Respectfully, that statement doesn't make sense. (Bear with me.)
Whether an investment is good or bad is orthogonal to whether the returns manifest in the short or long term: any calculation of investment return will discount value that we have to wait for.
Maybe you mean that the payoffs don't come in the short term. But, at a societal level, so what? As long as the expected payoff (appropriately discounted) is large enough, that's what matters.
That is my objection to FD3SA's comment that "Science has never payed off as a logical financial investment." and my objection to FD3SA's comparision to art.
Just because science's financial payoff is long term does not mean it isn't a good investment. To confuse the two issues is illogical, in an important way - which is what I'm trying to object to in FD3SA's comment.
I'd accept the argument that it is difficult to capture privately the return from fundamental science. That might be a reason why much science has been government funded. But, again, that's a different issue.
Governments should continue putting money into science because it is a good investment - it has a high expected return, over the long term. And, in fairness, substantial public money, globally, is still invested on this basis.
>In the current economy we favour short term gain hence why science is in decline.
How our economy is managed in the short term is a different issue from whether the long term expected return of science is positive.
Maybe you mean that the way our economy is currently managed, it doesn't make sense for private actors to do science. But that's a totally separate issue from whether the societal return is positive. Almost regardless of how the economy is structured in the present, if science has a high enough long term return, its still a good investment, at a societal level.
Someone could argue that funding art has not produced meaningful financial gain - if they were very 'practically oriented' (for want of a better term) - but the same argument would not apply to science which has produced huge returns.
Even today, there is a massive delta between the funding science and arts gets - find me the arts' equivalent of NASA or CERN - because the long term return of science is recognised.
Second, there are other specific mechanisms that favour short term over long term. For private companies, it's the desire of the stakeholders for fast return on investment. They want their dividends now, not in 10 or 20 years for now. We only live so long, after all —for now. For public funding, the government favour anything further than the next elections. That spurs decisions that may be rational for egotistical individuals, but which are nuts at a societal level.
If our society really were rational, it would fund basic anti-ageing research more, and basic AGI and FAI research much more. Not to mention cryonics. But what would you expect? Most people are sufficiently nuts to still believe in the supernatural anyway.
Don't worry, you'll learn to speak Normal again someday.
It's ironic that Silicon Valley owes most of it's existence to investments by the DoD and ARPA. Most of the technology we use in the electronics industry today would not exist except for all the public money that was poured into semiconductor research. Slashing public funding, especially of fundamental research, is one of the worst things we could probably do for our economy's future.
I mean, investing in a startup that has a value proposition based on real scientific research is the same as investing in the research in the first place. Am I right?
Not necessarily, if invest in science you're investing in the lab producing publishable results, without the constraint of bringing a product to market. The two don't necessarily overlap.
I therefore credit feudalism for the creation of the transistor.
What I am guessing went wrong here is that you are unfamiliar with sarcasm, so I'll help you out: I don't actually credit Feudalism with the invention of the transistor.
I'm not even going to go into how a lot of that spending, if freed up, would go toward endeavors that are not military related and to a parallel timeline we can never know. Imagine NASA with 10 or 50x the budget. Or NSF with 10x the budget, etc. We'd probably be typing this on a moonbase or on a cottage on Alpha Centauri's earth-like planet.
Instead we fawn over the peanuts that falls out of the elephants mouth and praise its generosity for feeding the hungry.
Transistors discovered by a private lab? A private lab with government funding? A private lab building on work done by individuals such as Faraday? Individuals who in many cases received government funding in the UK? Individuals who were building on work by Benjamin Franklin, a self-made and self-funded man? Benjamin Franklin, who doubtlessly was enabled by early work on the scientific method itself by Roger Bacon? Roger Bacon, who was supported by the catholic church? Roger Bacon, who built on work of earlier Muslim scholars?
If we want to play the "who gets credit" game, we need to decide beforehand how many times we are going to go down the "who funded who" tree, and "who researched the prerequisites" tree.
I'm not talking about politics; I am pointing out that you people are talking past each other because you all have different ideas of how to assign credit.
It's just now all the low hanging fruit is gone and you need million dollar labs instead of an apple and a notebook.
I think not.
It also has a lot to do with the Bayh-Dole Act and the privatization of research. For-profit research centers are one of the fastest growing areas of the economy. While more public funding would certainly help, the real issue is that under Reagan science became privatized. If there were less research but if it were still high quality, we'd be much better off.
and science in America in the 19th century was funded how? Because we sure didn't have the NIH, NSF, NIST, DOE, DARPA, or any of these organizations then.
Even into the 20th century: For example, most of Irving Langmuir's research (in both applied and basic science) was conducted privately, long before Reagan. Same goes for luminaries such as Gilbert Lewis, Josiah Gibbs, Robert Millikan, etc.
Are you serious ? It's so obvious that using Science is proving to be profitable for all industries using it. Life Sciences being maybe an exception rather than a rule, because of the complexity to address the issues at hand and the regulatory framework enforcing conservative approaches (we still categorize cancers by "location" rather than protein tracers which makes no sense at all scientifically). All the R&D going on in Google, Apple, Amazon and other companies involved in new technologies are driving innovation forward and proving to provide very tangible returns. The same goes with all the applied sciences done in the automobile industry, the aviation industry with material sciences, and I could go on and on since there is a neverending flow of examples.
While surprising this statement gets at something missing in science today. Until relatively recently basic science was not dominated by a big institution monoculture but by curious and creative individuals working in a great variety of situations often in near isolation.
I wonder if much of the effort to institutionalize and "professionalize" science has been counterproductive. I can't imagine most of the great figures in history of science thriving in the current environment. Much like healthcare is now directed by insurers, science seems to be increasingly directed by bureaucrats. Should we be surprised by the disappointing results and soaring costs?
I know this is a bit of a rant and I know times have changed. But perhaps something similar to the open-source revolution in software can get science back to its roots.
I think you have really hit the nail on the head with this one.
Public funding for science has generally been growing. It was cut a bit in 2009, but that's the extent of it.
http://www.aaas.org/spp/rd/fy2013/hist13pDefNon.pdf
http://www.aaas.org/spp/rd/fy2013/hist13pAgcyRes.pdf
I don't know where to find stats for private R&D - do you?
The scientific funding and publication system has problems that deserve scrutiny, but science itself is far more rational than nearly any large, human-maintained system I can think of.
When we resort to hyperbole like "science has lost it's way", we give a group of vocal, clueless idiots more power to undermine the most consistently productive engine for progress that humanity has ever devised. So let's talk rationally about the problems, but don't throw the baby out with the bathwater.
"Scientific journalism has lost its way" would be more accurate were it not for the fact it clearly never had one.
If you look at the typical application for funding, you'll see questions that basically prod you to explain why your research/students/etc. are exceptional/revolutionary/ground-breaking. Everything must look like a Nobel prize waiting to happen if you're going to have a chance at beating out everyone else making the same application (and exaggerations). It's utterly ridiculous! It's as if thousands of guitarists were auditioning at the same time and, in an effort to be heard, each has cranked their amp to 11. The result is a cacophony where even each individual sounds awful because of the distortion. If everyone dialed it down to 5 things would be bearable, but there's always someone willing to nudge it up to 6 or 7...
A nice long list of high profile publications is great to hype when your amp is set to 11. If you have published many papers in high impact factor journals (again, often by inflating the significance of your work), you must be worth funding!
Perhaps scientific funding needs to be awarded in a manner that is more... scientific. Heck, perhaps funding agencies should reserve a certain percentage of their funding specifically for reproduction of results. Currently, if you apply for a grant to check other peoples work, people doing original research will win absolutely every single time. Unfortunately, the preference for original research goes right to the very top of governments. Politicians want brilliant nobel price winners, not competent fact-checkers.
It wouldn't work for every field and area, but it could work for a significant subset of research.
I've done it a few times myself (partially because my work requires it and partially because I wanted to convince myself of the work I was using), and it was an incredibly valuable learning experience.
In general, a replication requirement might drive us to build replication tools for each science, which could be quite useful.
I don't understand what you're trying to achieve by saying this. Did I somehow imply that people should be excused for not making their work reasonably reproducible?
It my sub-field, it is utterly ridiculous to expect to reproduce work by running `git clone && make` or by getting a prepared VM. Computational biologists are not known for their systems or software skills.
Thanks for that link---it's always nice to see other people in the field concerned with this. There aren't enough of them.
That's how Reinhart and Rogoff's “Growth in a Time of Debt” spreadsheet error was finally unearthed, unfortunately that was after it had already been instituted internationally as public policy, but hey, you can't have everything.
http://www.dailyfinance.com/on/reinhart-rogoff-debt-GDP-spre...
"Requiring" has the same motivation problem - those who could require it, currently would rather require those students to do something that brings funding or prestige, so they won't.
In time, such a program would bring quite a lot of prestige as flaws are discovered and fixed in existing work. It's really the easiest and most immediate way to address this problem since it only needs to involve single institutions (whose faculty presumably care deeply about this issue).
It doesn't seem likely that journals will suddenly start valuing verification work. Similarly, politicians and funding agencies appear uninterested in actual science; they care only about their careers or immediate application to the politically popular cause of the day.
If the results are hard to replicate, or dependent on other causes, then usually that's when a project shifts or when the knowledge pool expands (for example chemistry is fraught with environmental effect dangers - fluorescent lights provide UV to reactions, your glassware has imperfections, the temperature and humidity of labs varies with climate).
Then again, we have the benefit of being able to "make" our own data so to speak, don't have IRBs to worry about, etc. This is trickier in other disciplines.
I think students should get course credit for it, instead of adding another requirement for PhDs, just to keep them from taking longer than they already do...
Science, at least in biology, is just like any business. Both are motivated by $.
The good news is that open science and the impact that the internets is having on science can help this problem. In my opinion, transparency in science will fix many of these issues.
1. http://www.theatlantic.com/magazine/archive/2010/11/lies-dam...
2. http://marginalrevolution.com/marginalrevolution/2005/09/why...
Marginal Revolution also throws a tip of the hat to Brad deLong and Kevin Lang's paper condemning econ journals for refusing to print any piece that fails to reject the null hypothesis: http://www.jstor.org/discover/10.2307/2138833?uid=3739936&ui...
Speaking of econ, this feels like a market failure to me. I'd be in favor of redirecting some portion of government research dollars towards an independent "validation" shop staffed by scientists who attempt to independently replicate submitted findings, and vow to write up all results (even detailing events that lead to the interruption or cancellation of an experiment). Findings that cannot be replicated by the validation shop should be viewed with extreme skepticism. Researchers would quickly learn not to fudge the results, and find more effective ways to control their own unintentional biases.
It wouldn't have to be government. It could be useful as a nonprofit, I just don't think it'd be sexy enough for anyone to support, despite the sort of urgent necessity of something like this.
Papers would have to provide access to their raw data and the code used to process it. You could then just run code(data) to generate the figures / results in the paper.
Two problems here:
1) assumes that the raw data is ok (which is a big assumption).
2) existing scientific code is largely terrible (completely imperative, no abstraction, no documentation, poor typing conventions), and it is very unlikely you can do
figures <- code(data)
This is what forced validation would aim to change though.
To really validate a paper you have to be able to recreate their code from a high-level description, then check that.
It might be that their high level description (usually in papers anyway) is correct but their implementation is flawed in some subtle way that peer review doesn't pick up.
Assuming a correct implementation this would be useful for anyone wanting to use the methods of the paper.
For example, I've spent the last 3 weeks coding a numerical solver for some equations in a fluid mechanics paper. Having the code for this available would have saved me figuring out all the quirks of the solution.
The walls of the ivory tower of science collapsed when bureaucrats realized that there were jobs to be had and money to be made in the administration and promotion of science. Governments began making big investments just prior to World War II...
Science was going to determine the balance of power in the postwar world. Governments went into the science business big time.
Scientists became administrators of programs that had a mission. Probably the most important scientific development of the twentieth century is that economics replaced curiosity as the driving force behind research...
James Buchanan noted thirty years ago - and he is still correct - that as a rule, there is no vested interest in seeing a fair evaluation of a public scientific issue. Very little experimental verification has been done to support important societal issues in the closing years of this century...
People believe these things...because they have faith."
From Kary Mullis, the Nobel Prize in Chemistry winner (and the genius inventor of PCR) in an excellent essay in his book "Dancing Naked in the Mind Field".
I made it through to the state science fair in 8th grade. It was based around magnetism, and after months of work, it turned out my tests just weren't sensitive enough to measure any difference in any of the electromagnets I built.
When I mentioned this to my teachers, they encouraged me to fix the results with a wink and a nod. Sure I could have turned in a "failed" project and maybe got a B, but I was an A student and there was no room for failure.
I'd love to see some data on how many high level science fair projects are faked each year.
People love negative results. But proving something doesn't work requires proving you didn't stuff up the implementation. That is amazingly hard, and about 10x more work then showing a positive result.
For example what you're talking about with magnetism isn't a negative - you can prove that an effect is bounded by the lower limit of accuracy of your instruments. Physicists do this all the time, because it's the correct way to phrase the result. I'll not to speak to the quality of your teachers though.
It really makes you wonder what percentage of what we "know" is true.
The Begley 'study' is impossible to assess, because they didn't report what the studies were, nor any of their methods, or anything. It's BS and hearsay, not science. Moreover, according to the Begley article itself, "The term 'non-reproduced' was assigned on the basis of findings not being sufficiently robust to drive a drug-development programme."
Nobody said that the purpose of every single scientific paper was to enable Amgen to go start a drug-development program.
There are many problems with the science funding situation, the glamour pub game, excess hype, funding getting sucked up by mega-projects, lack of open-access, inability to publish negative results, etc, etc, etc, but in general, it is not true that scientists are making up sexy results to get them into Nature and Science.
By cheating I mean... flat out lying. I don't know the implications of this (how far misinformation can get) but it seems like a wrong culture and attitude, especially for science.
At the time I definitely wondered how much of that went on in "real" science. I suspected (and still suspect) it's a lot more than is ideal.
Acknowledging, attempting to quantify, and then (some institutions) attempting to fix systemic issues in the peer review system is not an emerging crisis. We have to fix incentives, but we aren't about to see some fundamental tenets about to be overthrown here.
It isn't clear what the "big cost" is referring to. Certainly money has been spent on poorly-founded studies with fundamentally inconclusive results. If it instead refers to opportunity cost, fortunately we have the entire future of humankind to pick up what we might have figured out earlier.
For example, many of these high-profile, possibly erroneous (or occasionally fraudulent, it seems) Nature or Science articles are high-profile because they seek to address a contentious or long-standing problem in the field. When this happens, there are typically existing alternate hypotheses. It's much easier to get papers published or grants funded by seeking to test competing hypotheses than to simply try to verify an isolated study. It can also be easier to find weaknesses in an individual study by testing it in a different way, or against other models, or whatever, than by simply trying to reproduce it. Often, a single study might be impossible to directly replicate, or the underlying flaws may not be apparent until the problem is approached from a different angle.
Granted, this can take a couple years or even decades, but falsehoods (intentional or not) tend to become more apparent as their context becomes more clear.
Most papers have always been flawed, wrong, or not reproducible. There has always been pressure to publish--going back even to Newton's battles with Hooke over gravity, or Darwin's rush to publish On the Origin of Species before Wallace.
What has changed are the cultural expectations. Culturally, we've become spoiled by physics. We're used to the precision, speed, and accuracy of physics and engineering. Moore's law, the iPhone, incredible bridges, the 787 and 380 airplanes--they all just work, safely and reliably.
Note that the reproduction problems are most prevalent in chemistry, biology, medicine, etc. These are areas of science that are far more complex, and about which we know far less, than physics. It will take a long time, and a lot of failed research, to even start to approach that level of knowledge. Given the complexity, it might be impossible.
You could predict with decent accuracy how probable a study is to turn out to be true or false. Then you can use that information to decide whether it would be worthwhile to do more studies.
[1] http://www.ncbi.nlm.nih.gov/pubmedcommons/
[2] http://retractionwatch.wordpress.com/2013/10/22/pubmed-now-a...
[3] http://www-stat.stanford.edu/~tibs/PubMedCommons.html
USING MY EDIT WINDOW:
Some of the other comments mention studies with data that are just plain made up. Fortunately, most human beings err systematically when they make data up, making it look too good to be true. So an astute statistician who examines a published paper can (as some have done) detect made-up data just by analyzing what data are reported in a paper. A researcher who does this a lot to find made-up data in psychology is Uri Simonsohn, who publishes papers about his methods and how other scientists can apply the same statistical tests to find made-up data.
http://opim.wharton.upenn.edu/~uws/
From Jelte Wicherts writing in Frontiers of Computational Neuroscience (an open-access journal) comes a set of general suggestions
Jelte M. Wicherts, Rogier A. Kievit, Marjan Bakker and Denny Borsboom. Letting the daylight in: reviewing the reviewers and other ways to maximize transparency in science. Front. Comput. Neurosci., 03 April 2012 doi: 10.3389/fncom.2012.00020
http://www.frontiersin.org/Computational_Neuroscience/10.338...
on how to make the peer-review process in scientific publishing more reliable. Wicherts does a lot of research on this issue to try to reduce the number of dubious publications in his main discipline, the psychology of human intelligence.
"With the emergence of online publishing, opportunities to maximize transparency of scientific research have grown considerably. However, these possibilities are still only marginally used. We argue for the implementation of (1) peer-reviewed peer review, (2) transparent editorial hierarchies, and (3) online data publication. First, peer-reviewed peer review entails a community-wide review system in which reviews are published online and rated by peers. This ensures accountability of reviewers, thereby increasing academic quality of reviews. Second, reviewers who write many highly regarded reviews may move to higher editorial positions. Third, online publication of data ensures the possibility of independent verification of inferential claims in published papers. This counters statistical errors and overly positive reporting of statistical results. We illustrate the benefits of these strategies by discussing an example in which the classical publication system has gone awry, namely controversial IQ research. We argue that this case would have likely been avoided using more transparent publication practices. We argue that the proposed system leads to better reviews, meritocratic editorial hierarchies, and a higher degree of replicability of statistical analyses."
Like all other sectors, scientific research can get inbred and peer review corrupted as a mechanism. It's similar to a character/job/performance reference: "We want to hear what other people say about you," can be problematic when the folks talking are untrustworthy--yet they wield credentials imparting trustworthiness. Peer review only worked when the majority of peers were rock-solid good scientists. (when there were fewer scientists, with personal reputations and the discoveries to back up their reputations) Wouldn't it be great if Pierre Curie, Darwin or Tesla were doing peer reviews? (or men/women of similar caliber)
At leading research schools (they aren't all universities), falsification of data exists at the student and professorial level.
Funding is definitely cut at NASA, whereas "sexy" research funding is increasing. We need excellent researchers across all areas of expertise. And we need increased accountability and transparency. And more funding.
I argue that what is most needed is increased scientific literacy at the level of political leadership and the general population so that findings can be accessible/understood/evaluated on a more concrete level by all.
Coming pathogen shifts associated with climate change, and extreme weather events, etc. alarm people. And people want to be able to trust science and to trust science reporting.
1. Reproducing work is a waste of resources. Use the money and researcher hours to develop tools that are more reliable, cheaper, and easier to use. A major reason why no one reproduces experiments is that the initial work was very difficult. Let's invest in technologies that make science easier. Reproduction (of experiments) should be done in high school biology classes.
2. Technical reproduction is rarely done, but conceptual reproduction is common. Findings in the literature become incorporated into disparate subsequent hypotheses tested by many other labs. If something doesn't add up, this will often increase the impact of the paper and eventually be addressed through experiments to resolve different models of the phenomenon.
3. There is no widespread fraud in science. Your academic career rests on your integrity. When I publish a paper, I do my damn best to make sure it is accurate. My reputation relies on it. When scientists continue to publish results that are false or fraudulent, they become discredited within the community. All graduate students in the life sciences are required to take a class on the ethics of science.
4. Publishing is a bitch and a source of real rot within the community. Fortunately, many researchers and academics recognize this problem and are addressing it. Look at the new journal eLife, or open access journals, or the increasing interest in arXiv.org (moving to a publishing model closer to that found in math and physics, which appear to be healthier research communities than life sciences). As experiments become more technically advanced, the expectation for methods sections have increased, not decreased.
5. People want to commercialize scientific findings that are relatively new - it's obvious that is risky! Why put the burden back on (under-funded) scientists? Drug companies are the ones that would benefit financially. Or wait until the phenomenon is better understood. Notice they're talking about drugs for complex diseases like cancer, metabolic disorder, etc., not Mendelian diseases. It's as if people complained that they couldn't get their lasers to work using a 1917 understanding of the physics of light. But Einstein demonstrated the fundamentals! Why did it take 40 years to make it work in practice?
Therefore, when I see an article like this with such a broad, generalizing headline, I just think it's click-bait. Lost it's way? I've read some absolutely terrible papers. "Theory of the Origin, Evolution, and Nature of Life" by Erik Andrulis is an excellent example of such unfathomably speculative garbage. I've also read a huge number of well-done papers on topics in aerospace engineering and materials science. It's always on the reader to re-produce experiments if they depend on the result, to understand the paper correctly etc. This is what my professors did. If part of the community is circle-jerking, let evolution run its course. We used to treat Aristotle as canon in the western world. Obviously things get better over time.
Skimmed article. Old news. The fact that someone is raising the flag, saying "there's a lot of low-hanging fruit to use to establish yourself as a more accurate researcher," just means we will see more of such review activity, making the title seem inaccurate. You never know when you can free yourself up an adjunct professor position in exactly your preferred field of research.
The major reason for that is, however, that most people in the medical & biological area are rather lacking a profound mathematical education. There are cases where papers get rejected because they are too mathematical.
Physicists are generally looking for very very small effects, hence very high n (higher n = higher power = more sensitive to treatment effects). This doesn't mean lower samples sizes are insufficient for other areas of research.
Anyway, the real issue is over-reliance on convenience sampling.
[1] http://sph.bu.edu/otlt/mph-modules/bs/bs704_probability/BS70...
There is also the criterion of "impact factor", for papers and publications. It's very similar to a "Karma" system as used by HN, Reddit, etc., and it has many of the same problems. Imagine a system where you have to choose between doing research that might be vital but probably won't, versus something safe and predictable that ensures that you get paid next year.
The problem is not so much science as the management and funding of science, which have been infected by the same managerialism that causes so many problems in big government and corporate projects.
The way funding works, in particular, means people publish stuff-that-will-get-references with a similar attitude to web start-ups iterating their code, (by which I mean too damn fast and not listening to the peer reviews.)
I'd love to see this change, but I don't know how central agencies can easily/affordably work out which research(ers) to fund and which to cut. As others have said, by definition we don't know what work was useful/valid/critical until many years later.
Where can we find the list of these studies?
http://www.nature.com/nature/journal/v485/n7396/full/485041e...
The irony is thick:
1) They were unable to reproduce the experiments from the papers alone, and this is considered normal.
2) The "scientists" required them to sign an NDA before helping them to reproduce their published results.
3) We're left here drawing conclusions from anecdotal evidence from a study that cannot be reproduced.
Luckily, science is self-correcting so we need not be overly concerned.
It would be very cool if someone came up with a unit test framework for various fields of science. Then we could make reproduction unit tests a requirement of publishing, so anyone with the proper equipment/framework could sync and run the tests themselves.
Or are things really this bad?