> When you spent an entire week working on a test or experiment that you know should work
I thought the whole point of doing experiments was to challenge what we "know" so we can refine our understanding?
I thought the whole point of doing experiments was to challenge what we "know" so we can refine our understanding?
In reality, scientists are highly motivated (i.e. biased) individuals like anyone else. Therefore science cannot be done effectively by individuals.
The system that derives truth from experiments - the actual scientific system - is the competitive dynamic between scientists who are trying to tarnish each others' legacies and bolster their own. The scientific method etc. primarily makes scientific claims scrutinizable in detail, but without scrutiny they are still highly liable to produce false information.
Personally, I think the “highly” in your statement is quite over exaggerated. Humans can be convinced to produce bad science, for sure, and there are even journals set up by religious orgs that specifically exist to do just that.
But at the same time, science landed humans on the moon.
Except that the entire point of the article here is that it's not exaggerated.
> But at the same time, science landed humans on the moon.
Cherry-picking a highly successful, well-known example doesn't prove a point.
There must be hundreds, if not thousands, of successful scientific discoveries that went into something as complicated as the moon landing, and if you still don't think that's convincing, just look at the world around you - which looks just radically different from the world of, say, just a couple hundred years ago.
As if our lifespans and quality of life haven't been drastically improved by modern medicine.
I mean, we can cut people open and replace entire parts of them and they're fine. They don't even get sick anymore - thanks germ theory and aseptic technique! Do you not understand how much of a marvel that is?
Before that, people used to get cuts and scratches and just... die. We can now fully rummage inside an arbitrary person's internal organs.
And don't even get me started on long-term illnesses. High blood pressure and cholesterol has been killing humans since forever, and we have medicine that just fixes that. And now, we're getting medicine to rewire our brains to prevent addiction in the first place (semaglutide)
Tell me of a better method to get to the truth. Go on.
That said, wonder drugs are few and far between. The GLPs are at least a once-in-a-decade breakthrough, so that’s probably most of the noise you’re hearing (there are a lot of brand names already).
No one is under the illusion it’s perfect or ungameable. A drug slipping by every few years is bad and often tragic, but IMO nowhere close to indicative of a systematic problem. It is a system that is worthy of a high degree of trust.
Shouldn't we expect some small percentage of failures in these processes given that they are driven by statistics and confidence intervals? Is that even a failure of the process, or is it a known limitation given how much resources and time we are willing to allocate to the discovery process?
As patio11 says, the correct amount of fraud in a financial system is not zero, and the correct amount of false positives in drug approvals is not zero.
Yes, and this is really solved at a more local level. Doctors aren't prescribing new drugs like candy. They, too, are skeptical of their success and will reserve those prescriptions for the most desperate cases. Over years, we (and the doctors) learn how effective these drugs are and what potential side effects they have.
Cardiovascular safety was tested in the original trial. It passed. Nothing in the data during development suggested it was an issue. But trials can’t detect everything.
It wasn’t until it got to market did a safety signal pop up. Then retrospective analyses of large data sets proved it.
Republicans and Russian bots WANT you to hate science and academia and they have frequent pushes across social media platforms to make sure you do.
That was engineering. Closely linked to science, but not the same process of inquiry.
It's not. Markets are good at that. They're actually competitive. Academia is good at producing enormous volumes of documents that claim to be information, and may or may not be if you test them. It's "competitive" in a weird way where people don't compete over what's actually true but over who can convince central planning committees to give them money, which is very different.
You can't separate the production of correct information from the production of goods and services. They're inherently intertwined, the attempt to separate them is how we ended up with such a polluted scientific literature. Having formed a hypothesis as to what is true, you have to test that out in a robust way where you can't easily cheat and you can't easily cheat yourself, even when "yourself" refers to the vast institutional structures that employ you. In other words you need a system that stops you cheating even if that's what would please your boss, your vice chancellor and ultimately the President.
We only seem to have one such system and that's markets. If you cheat then the product or service you provide will be based on beliefs that are false and - eventually - customers will abandon you because the thing you're selling doesn't solve their problem. This is effectively a referendum of the customers. There is no such feedback loop in academia. There are attempts to approximate or emulate it with things like peer review, but they're all shadows of the real thing.
Meanwhile you can encourage cooperation even between competing entities in lots of ways, and it often emerges naturally even in the absence of any specific social policy. Open source collaboration is one obvious example in the tech sector, patents are a more formalized system.
> The system that derives truth from experiments - the actual scientific system...
Yes!
> ... is the competitive dynamic between scientists who are trying to tarnish each others' legacies and bolster their own.
Hm. To some degree, sure, that is one dynamic, but (a) this leads to/presupposes a truckload of perverse incentives and (b) this is not inherent in the system if we rearrange incentives
Of course the implementation is far from perfect. For example, the interaction between impact factor and grant funding produces pressure toward ideological conformity and excessive analytical “creativity”. But the underlying principle of competitive scrutiny is probably a desirable one.
Cooperation is also an extremely fit behavior in natural selection.
If someone has a large bag of money laying around the plan is this:
There are lots of companies that will run material A though machine B for you. There are a lot of science machines. One is to put a lot of them into a large building and make a web page where one can order the processing of substances in a kind of design your own rube goldberg machine.
It can start with all purchasable liquids and gasses, mixing, drying, heating, freezing, distilling etc and measure color, weight, volume, viscosity, nuclear resonance etc, microscope video, etc. Have as much automation as possible, collect all the machines. A robot cocktail bar basically.
Work your way up to assembling special contraptions all ordered though the gui.
Jim can have x samples of his special cement mixture mixed and strength tested. Jack can have his cold fusion cells assembled. Stanley can have his water powered combustion engine. Howard can have his motor powered by magnets. Veljko can have his gravity powered engine. Thomas can have his electrogravitics. Wilhelm can have his orgone energy.
or not... hah....
If any people are involved they should not know what they are working on.
It wont be cheap but then you get an url with your nice little test report and opinions be damned.
But if you want to without human error/bias there is nothing close to removing all the humans.
Things that are controversial, unbelievable or unlikely may have big implications and risking your career on it is usually not a good idea - for you.
Though automation one might drive the prices down to make the brute force approach viable but with somewhat intelligent machines one could also make educated guesses in volume.
You could auto suggest similar experiments while the researcher types their queries complete with prices.
The original question was: How can we do more research without increasing the number of scientists.
I’ve never fabricated numbers for anything I’ve done, but there certainly have been times where I thought about it, usually after the fourth or fifth broken multi-hour test, especially if the test breakage doesn’t directly contradict the hypothesis.
Although, contrary to what I was taught in elementary school, most of the experiments in the physics department of my university didn't even really have a hypothesis. They were usually either of the form "we are going to do this thing, and see what happens", or "we're going to measure this thing more accurately than anyone before".
An example has been times where I really want to use a certain concurrency style, and I'm convinced that it should be faster than the way we're doing things before, so I will write a few non-trivial tests to make sure that's right, and I'll get inconclusive numbers.
Of course, that is still very frustrating if you are on a tight deadline and all you have is one thing you know doesn't work.
A significant result in an experiment, according to Fisher, is just an experience to add to the mental pros-and-cons list. It is not defintive proof of anything.
Plus these days there's a lot of pressure to run universities more like businesses. To eat, academics have to hit certain numbers, so you see behaviors common in business like faking the KPIs.
If the claim is false, though, you can still sometimes get research to support it. If you or the researcher stands to profit from the false claim, then there is a conflict of interest.