> I believe 80% of science wrong AND not reproducible
Maybe that's the case in your area of research. In mine (math, physics) it definitely isn't. So I would be a bit more careful about the wording here. ("80% of science" – what science?)
Maybe that's the case in your area of research. In mine (math, physics) it definitely isn't. So I would be a bit more careful about the wording here. ("80% of science" – what science?)
P < 0.05 is outdated at this point, for anything truly ground breaking p < 0.005 or < 0.0005 is probably a better choice, and even then I would ask "Where did you get your dataset from and did you combine (!!!) datasets from multiple orgs."
The methodology is deeply flawed, given that you haven't stopped accidental p-hacking (publication bias) and deliberate p-hacking (researcher data mining, e.g adding endpoints after looking at results), which compounds to create fake results with astronomically tiny p-values.
You can only trust single, extremely large, canonical RCTs which were announced in advance, and in which you are confident there is no survivorship bias in terms of the possibility that this RCT would have been cancelled had the results been thought to be negative halfway through.
Epidemiology (victim of researcher p-hacking and impossible to deal with confounders) or meta-analysis of RCTs or single small RCTs or RCTs that weren't announced in advance (victim of publication bias) should be taken with a grain of salt. If you accept conclusions drawn from these things at face value, be prepared to accept anything, because the methodology you've accepted is proven to be easily capable of demonstrating any fake phenomenon as true.
There are so many ways that are studies can be flawed either accidentally or purposefully. I truly assume most studies are not purposefully fraudulent - people, by and large, are honest... but I do believe there are enough problems with our current methods that most studies are not truly accurate.
Having a publish clinical trial regimen - created before your study starts - helps with a lot of this. It answers how you segment patients, match patients, handle patient dropout, and specifies what you are trying to compare for outcomes.
Is it the end-all be-all? obviously not. I don't know what the true answer is.
Its actually my job.
What are you designing new experiments for? Are you taking results in one cell like and condition and trying to transfer them to new cell lines and conditions?
Are you trying to reproduce the same data in the same cel lines/organisms?
There's such a huge wide array of what your could be taking about, but based on my experience in life sciences what you say does not seem the least bit realistic, in my first interpretation of what you are saying.
Also its worth noting that I am not claiming my experience is the end-all be-all. I am stating that, from my experience, I have incredible distrust for many studies that are published with 'amazing' results until peer reviewed preferably on disparate datasets.
My experience is based around outcome based studies of the effect of drugs/treatment/regimens in oncology and oncology adjacent fields. This includes drugs treated alongside traditional cancer regimens to assist with managing adverse events and toxicities.
Other fields may not have this reproducibility problem. Mine does. Even if the study design if perfect, and I can't imagine most are, the data itself can be questionable.
Consider - what dataset would you use to identify if patients taking keytruda had a higher incidence of high blood pressure?
You can use data from an EHR, licensed for deidentified studies, but EHR data is a burnt down trailer park of questionableness and its use in studies has been laughed at in many conferences.
You can use data from individual enrolled patients (for a clinical study) but then the cost is extremely high vs a non interventional observational study using other data methods. The value of the data is likely to be higher, but since it costs more to collect maybe you are only in a few regions that may have a higher prevalence or incidence of this anyway. Troublesome.
What about insurance data? You can get it cheaply, if you have high blood pressure GOOD doctors are likely to medicate you with a drug meant to treat it, and you can get it across the country. Seems good right? And it is, as you can generally draw an implication of high-blood-pressure->treatment-with-drug-x. So for a yes/no study it can help, but what if the base condition causes high blood pressure and we want to tell if the drug causes a HIGHER amount of high blood pressure than others. Insurance data by itself may not be enough to tell this data.
So what do you do? You are stuck with no great answers.... and this is assuming your study design is perfect.
So you buy multiple datasets in some third party health marketplace, and someone gets the great idea to combine the datasets to increase the n value. Well, too bad those datasets have a high overlap. So you have attributed a higher power to the study than is relevent.
I hope this explains more about the concerns I have. Though, I suppose, it may mean that this account is now dead. I will have to think further. Anyway, hope you have a great day.
You may disagree with the experiences I have had as it may not be in alignment with your thoughts and ideals, and it still doesnt make you a bad person. It doesn't make me a bad person either.
Even then, I would worry that the results may be caused by some confounder in the original dataset/design instead of something you can trust.
One factor contributing to this. In natural sciences, you take other people's papers as truth and build on that. In theoretical physics on the other hand the _first step_ is you reproduce their results.
In theoretical physics, one of a researcher's main concern is to not to make wrong claims. Granted this risk aversion has its own downsides, but the upside is that he is extremely careful on what he takes to be true. So it's not true that not reproducing the research you rely on has limited benefit. It has the huge benefit that the researcher convinces himself he's not introducing wrong premises. Research is already difficult enough without making mistakes in this silly way.
If your research is mathematical or involves a lot of engineering, you are building on earlier results. In many cases, your own results will depend on the correctness of earlier results in a measurable way. You end up replicating others' research without even trying.
Empirical research is harder. You cite earlier results, but there is no clear connection between their correctness and your results. Especially if effect size is small. Earlier research has more effect on the framework you use to interpret your results than on the results.
Academic researchers rarely replicate others' results, because it's expensive and not particularly interesting. People typically come to the academia because they want to work on something they personally find interesting. If you want to have experienced scientists working on something administrators tell them to do, you better pay industry-level salaries.
As for physics, well it depends. Laboratory physics has produced the finest predictions in any science by several orders of magnitude. Quantum Electrodynamics is freakishly accurate. On the other hand it’s hard for me to see cosmology, to gently pick on an easy target, as more than extremely well researched and plausible science fiction. Then you have particle physics which has excellent laboratory equipment and produced fantastic results, but which has, in the opinion of at least one elite particle physicist personally known to me, perhaps painted itself into a corner. The Standard Model is good enough for government work, but nobody believes it’s the best possible theory.
For physics, it depends.. is it just "applied math" (so, verified easily), is it a CERN-type (LHC,...) experiment (hard to replicate, unless well.. you work at CERN), where many many people process the data, or is it something that is done only in your "lab", and hard for others to replicate.
On the other hand, finding thousands of patients and running a study is practically always hard and expensive.
But what this logic fails to consider is that people who graduate from Harvard aren’t successful simply because they went to Harvard. Their success comes from many attributes like their intelligence, etc that advanced courses are designed to separate the cream.
So then they organize and legitimize their power (removing merit and replacing with lotto or affirmation quotas) by claiming the existing system is racist. When you ask for specific examples they respond that it’s “systemic” and although no one can detect it, it’s imbued in everything. The solution is “anti-racism” which means to make up for past discrimination by systematizing present and future discrimination. This is why your HR department probably has a commissar on it now. They might call it DEI Officer or sone other bullshit job title.
This is what social sciences have contributed the last 40 years.
"This is what social sciences have contributed the last 40 years." This statement implies that's the entirety of what they have contributed which is false.
Anyways - you seem to like to take this line all the time - "where's the evidence". It's everywhere. It isn't my job to keep you informed of the world you occupy. Either willfully or not, your inability to keep up on developments isn't an excise to demand "sources" when you have access to the same search engines as everyone else. This information isn't difficult to find.
Yes, some people use "where's the evidence" as a conversational gambit to try to shut down discussion they don't like. (And if you provide evidence, they may say "that's only one source, got any others?") And if they're being dishonest in asking, there's no point supplying the evidence they request; it is useless to try to have a conversation with those who will not listen.
On the other hand... when you make a claim, the burden of evidence is actually on you, not the other person. And if you say "you have access to the same search engines as everyone else", well, that's true. On the other hand, one person writes a post, and ten people read it, or a hundred people, or a thousand. Making the thousand do the searching, instead of having the one writer do it, is really inefficient.
This leaves you at "do the work of providing the evidence, but don't feed the dishonest trolls", which is... well, at best it's not very actionable advice.
I told you I was torn...
I mean, affirmative action is literally this in practice.
By proposing a math curriculum that requires teaching all students the same material, regardless of their ability, with the aim of increasing social equality.
I'm OK with that aim; but I know from my own experience that trying to teach calculus to someone that's not ready for it isn't just a waste of effort, it's disastrously counter-productive (I totally fell out of love with maths when I was taught integration, failed to "get" it, and my well-regarded teacher didn't get why I didn't get it).
My understanding is that nowadays in UK state schools, maths is largely student-paced, using worksheets; they've given up on trying to get a whole class of students to all understand the same stuff. That's partly because a set of worksheets is much easier to come by than a good maths teacher, of course.
I'm not a maths teacher, and I don't know enough about the California curriculum arguments to have a view.