The Karikó problem: Lessons for funding basic research
statnews.com
statnews.com
For an even more glaring example, even the best of the incredibly selective IITs don't make it into the top 500 in the US News global ranking list.
Nature and others have released rankings that are a bit normalized by size, and many niche institutions such as Cold Spring Harbor or the Austrian IST are near the top. This was discussed in an old HN post, which is quite interesting to read through: https://news.ycombinator.com/item?id=20231432.
ARWU subject ranking is pretty good, though. Rankings that are not subject based are hard to interpret. For example, many smaller Ivies have excellent departments in topic X but are not so good in topic Y. They can't cover everything. But if you trust the ranking and go there anyway to study topic Y, you might be disappointed.
However, funding is really biased towards big or famous institutions. In my experience, it's really easy to get tons of grants from a place like Oxford or Cambridge. If you move to an excellent place that is less known, say a university in the top 30-50 of the world, it gets a lot harder even if your proposal is identical.
For Szeged, the university is kind of old but also only appeared in it's current form in 2000.
It's also called Szegedi Tudományegyetem.
It has multiple colleges inside it, so it may depend on how well those are represented in the data.
Authors using acronyms more makes identification less likely, each of the colleges have acronyms.
Finally, where authors publish makes a big difference - not all publishers open their data up and even those that allow access to some raw data it can be less reliable to extract. So if you publish in places where the publisher does not make the affiliations easily machine readable your work may be counted less in many of these automated rankings. Pubmed (not a publisher but a key way a lot of people get some of this data) didn't have affiliations for anyone but the first author until some fairly late date.
Either automated mappings are used for a lot of work (since there are tens of millions of unique affiliation strings), or the source material is heavily curated (so for example only the "top" N journals are looked at).
source: worked on affiliation mapping at Dimensions in a previous life
The broader issue you're mentioning is spot on, though: what are we incentivizing, really?
Things are so broken in academics, at least in the US. I'm not sure what is better but the current system just isn't working.
I studied undergrad in an IIT faculty that's considered the top in its field in India by reputation of the professors and I worked on two research projects with PhD candidates there. I also spent one summer working on a research project at KTH in Sweden. I must say the experience in terms of research quality, capabilities of the PhD students, facilities, support systems etc was night and day. Due to this in the IITs, the top undergrads leave the country to pursue post-grad and the folks left behind are often the ones who didn't get admits with financial support outside. Very often I observed the undergrad students assisting with the research understood things better than the PhD candidates themselves and all these makes a difference in the research quality and impact.
Now, if you rank university programs in terms of % change in life circumstances, IITs will very likely come at the very top. Because the entry is entirely meritocratic with minimal reservations and even reservations are held to academic standards - seats go vacant rather than being filled with less qualified students - a lot of students from poor families get boosted solidly into upper middle class with an IIT education including many of my own batch mates.
(FWIW I think ranking schools like this is generally harmful, and even "objective" rankings will still have plenty of biases baked in)
I am half-Bulgarian, but for all practical purposes I am a Czech, though none of my grandparents actually were (the rest were Slovak).
The same person explained that a 5% wage gap between women and men in a certain field needs immediate action but it's 'fair' that there's a 70% wage gap between his salary and the salary of someone from Eastern Europe (working in the same role at the same company).
Oh well.
I needed to master a foreign language for reviewers to not reject my papers because "bad English". I have had applicants to PhD positions reject my offer and go to objectively worse US groups with less productivity by any metric because "my parents want me to study at a US university" or "your group is good but I think that I need to go to a high-ranked university to improve my prospects" (not to speak about how many applicants I get vs. how many an US professor with equivalent track record would get). I spend a lot of hours on things like grading that US professors have TAs do for them. I have several times seen people cite "Person from Stanford/Google/etc. (2013) proved..." when I actually proved that in 2011 and that person was just citing my result.
And still, I have to listen to female, white, native-English-speaking professors from elite US universities, lecturing me about how they are discriminated because some study said that women get cited less often, etc. and keep a straight face.
A possible way forward is to simply give out grants randomly to all those that pass some minimum score. After enough filtering, all grants are essentially indistinguishable in quality.
The issue though becomes that the score gradually stops being minimal as more and more requirements get added in over time
Definitely a controversial idea and would pose huge challenges to the system we have now. We'd have to confront the idea that a Harvard pedigree or an R01 NIH grant aren't quite the strong indicators of success that we think they are. We’d also have to consider that brilliance can be found in places we haven’t looked more often than we think.
> After enough filtering, all grants are essentially indistinguishable in quality.
I don't believe this. You can always soften incentives (and thereby prevent wasteful gaming) by adding random noise/dithering to your scoring function, but a threshold-only approach seems quite blunt to me.
The problem is that impact by reasonable metrics is correlated about 0.30 with grant score, so it's difficult to predict what will be successful. (Note that impact itself is controversial, given self-fulfilling fad dynamics in research -- FOMO popularity spikes and so forth).
One of the biggest predictors of grant success is having previously published with people on the review committees.
The idea is that you randomly fund research because it's difficult to know what will be important, but try to balance that against grants that might be poorly conceived to begin with.
> I don't believe this. You can always soften incentives (and thereby prevent wasteful gaming) by adding random noise/dithering to your scoring function, but a threshold-only approach seems quite blunt to me.
These assessments come from statistically investigating grant scoring and success. What you find that 90%of grants are essentially scored the same, within the noise. In these situations what typically happens is that previous grant success will determine current success, i.e. the person previously got a grant so what they are doing must be good (the "safe" bet). The problem that creates that you create empires of people who get a lot of grants based essentially on one luck of the draw.
The biggest issue is though not even that you miss some good people with good ideas, but the incentive structure you create. Because there is an expiry time on the benefit from previous success and it is research area specific, what happens is academics have to write grants constantly and you can't really change fields.
Academics spend 30% of their time applying for and reviewing grants. None (hardly any) of the people who are very successful with getting grants are doing research themselves any more. I suspect this is also the reason why most Nobel winning research is done in people's 30s.more senior academics don't have time to do the research (and can't take the necessary risks).
I think a system where grants are only given to a small minority is defective and wastes everyone's time, no matter how the minority is chosen. The solution has to be to give some reasonable funding to anyone employed as a researcher - which, depending on how much you value science, can be achieved by increasing the funding or by decreasing the number of researchers (I'm an academic myself so I believe in the former, but I might be biased).
The variant you mention would be better than pure randomness, but at least for me, it wouldn't work either. First of all, in order to become "excellent" at research you typically have to start at "good enough", so the "excellent" category would also be hugely conditioned by the randomness - how do you build up an excellent CV if you repeatedly lose the grant lottery? Secondly, although incentives wouldn't be totally lost, for me it still wouldn't work psychologically. I can come to terms with the fact that there is a large random factor in grant reviewing, it's not something I like but that's just life and very difficult to avoid, but I wouldn't be able to accept seeing the researcher next door scoring a grant while I'm left without it due to a pure lottery, to be honest. I think I would go to industry.
PS: regarding what you mention about marketing, something I would do would be to evaluate people (CVs) rather than project proposals. Proposals are indeed very, very biased by marketing and by how boldly researchers can make grandiose claims (which is likely to be correlated negatively with research integrity). And in CS they often don't even make sense... for example I work in NLP, large language models like BERT came in 2017-2018 and revolutionised the whole field, if I had followed the text of my ongoing grants to the letter I would have been doing obsolete research. (I can see how in other fields like physics, etc., this may not apply).
There will always be people at the margins of funding, no matter where those margins are. Sometimes those people will hit it big for reasons that span persistence, cleverness, collaboration, serendipity, luck, and more.
There are some arenas in which widespread funding of small actors works well and others in which decades of focused investment can yield huge breakthroughs.
Overall, though, if you like the outcomes from basic research and want more of them, fund it.
It can also be somewhat wasteful, depending on what gets funded. Research that generates "very large returns" is probably quite rare and special, in a way that makes the whole notion of "basic research" somewhat less than meaningful as a target for funding.
Part of the problem I think is that there needs to be some healthy acceptance of risk in research. I'm not sure how you draw the line all the time between "things that are just bad ideas" versus "things that are unusual" but if you're always funding the sure bet you're not going to get anywhere.
The principled approach is to correct for that by looking for things that ought to be popular by current standards, but are nonetheless underfunded. Yes, this is hard - it's literally trying to beat all other grantors at their own game. It's also supposed to be hard. There was no reason to expect that "funding good work in science" would have an easy, painless solution.
The basic premise boiled down to: Scientists already want to work on valuable things, they just have different priorities. Understanding unknowns is motivation enough for them. But this makes the impact of their work also unknown, and scary to people who see that motivation as wasteful.
Making them justify ahead of time why their work will be useful, to satisfy the risk tolerance of financiers, means that they need to reduce that unknown before they even start, and potential impact with it.
If you just want to poke a stick at some lesser known Y, it's harder to appeal to those who want to understand potential returns on their investment.
And it seems that there was a time that predictability, and even returns, was less of a priority. I can't recall the podcast (maybe Planet Money) but they mentioned Vannevar Bush pushing for the government to act as a patron for sciences, and to take on risk that private investors would avoid. Like, spend up to find a new X, then let the private sector step in and develop the X-likes.
If you want non-wasteful research, doing only "safe" projects with a clear path to expected results then you are simply committing to never fund any of the research that has the potential to yield very large returns, and not getting those returns. That is the key point - that attempting to eliminate wasteful research is counterproductive and results in throwing out the baby with the bathwater, so to say. It has a lot of parallels with VC funding, as with basic research just as with startups you can't tell which projects will "strike gold" before the work is done; you can and should do some basic diligence but if you throw out uncertain projects, then you throw out all of the really good ones as well; all the projects where you are quite sure about the results and can know that they won't be bad are the same projects where you can also know beforehand that they definitely won't be groundbreaking.
Just to take one example: at one time, you had a university, someone applied for a tenure-track job. The idea was, if they passed the tenure hoops, had the approval of their peers, etc. the university was essentially "funding the person". That was the idea: the public (or private benefactors) trust a university to appoint faculty who will identify people who are successful with worthwhile research. That is the idea behind tenure at a university, at some level. You as the public entrust the state to fund research, with the idea that there is some benefit and risk attached to that benefit.
Now, though, this idea has broken down, in that the current paradigm is for the university to find people who will get funding from the federal government, to fund their projects. The university is no longer "funding the person" at some level, it's passing that buck on to the federal government, and letting the federal government decide. So while, at one time, we had a very decentralized "fund the person" model (in the form of individual states and universities deciding who to fund), that's started to fail, in favor of a very centralized funding model where we basically rely on the federal government for funding everything.
There's just not enough options for people wanting to do research; it's too homogenous and too narrow in terms of incentive structures. It's like anything else: if you set up a system where there's one way to do it, you're going to incentivize taking advantage of that system.
I remember ~20 years ago hearing about nanopore DNA sequencing, and wondering if it would ever work (it does!).
I would have definitely funded both of these people, but I wouldn't necessarily have funded the projects. I knew they were brilliant, but I didn't know exactly where it would lead or if it would work.
The key really is to break out of the current "big famous lab" approach, IMHO. Or at least have two tracks of funding. I'm not sure about how to fund people not projects in a systematic way, but we definitely need more random shots on goal and more stochastic exploration of the space of possible research areas.
It's astonishing that we have, essentially, a single "one-size-fits-all"[0] mechanism for funding biomedical research. We should have a bunch, ranging from "trust me, I'm a genius" to "you've never heard of me, but the data suggest this will work." There should be mechanisms where tons of preliminary data are required, but also mechanisms to generate that prelim data. There should be mechanisms for trainees, but also to keep experienced people in the field.
[0] Essentially, the NIH's R01. Even that needs work because inflation has chipped away at what one modular budget can support.
There's also been a rise in "but science" projects in bio, starting with the human genome project, that mirrors some of the big science that happens in physics. So adding in that track of funding there are kind of three funding tracks that are necessary.
>I remember ~20 years ago hearing about nanopore DNA sequencing, and wondering if it would ever work (it does!).
These two things are very different to my eyes.
Nanopore sequencing seems obvious to me as an electrical engineer. The basic problem was noise and control--which you could probably solve with computes and Moore's law. You can fund it, but it feels like something you could leave to industry as it seems like it's engineering more than science.
CRISPR is non-obvious. It's more in the realm of Taq polymerase--it's interesting but, so what? Well, Taq polymerase would become the keystone to PCR about 15 years later.
I'm much more interested in figuring out how to fund the things that later become building blocks rather than pouring money on something that industry is likely to do anyway.
Industry will only ever fund things that are easy to monetize in the very short term. Translational research can be just as important as basic research, so both should be funded when they are plausibly in the public interest, regardless of any private returns.
Usually. ASML worked on something for two decades, and now, against all odds, EUV works :)
But, of course, it is notable because it is far from the normal case. And it was, more or less, an engineering problem, which required some support from scientists.
The signal processing is still quite significant, and I hear that if you don't have a compute box that's set up correctly, you can't get data to and from the GPU quickly enough for base calling (no personal experience, just heard friends griping about early experiences). So even with todays compite tech, the signal processing is pretty heavy duty, 2000 era hardware and algorithms would have far far less accuracy.
Which is all to say that yes, now it works, and my uncertainty was perhaps unwarranted. I'm super impressed that it works, and I've learned a ton of humility about guessing what other people can figure out how to do. David Deamer is a visionary, and all the people that worked so hard on it afterwards deserve major kudos.
Diveristy in funding sources is important and should be encourgaged. The main issue with these alt funding initiatives is that they lack scale and the sustainability is uncertain. I heard the head of research at a very large and storied biotech dissing Sean Parker's $500 million cancer immunology initiative as being a drop in the ocean, to give you an idea. Life science research is incredibly expensive and wasteful, like setting money on fire. Yet medicine is still so backward we need at least 10X more to really dent human suffering. If you disagree come to my clinic and see what you think.
This is common outside the life sciences, too, FWIW. It seems that we're doing a very poor job of organizing the existing body of research in ways that could benefit the field as a whole. Not to mention that older papers/monographs/etc. are generally not even digitized, which makes them essentially inaccessible by modern standards.
Everyone has their own way of interpreting "fund people, not projects" but my take on it is that it would basically be "basic income" for scientists and engineers. Make a lot of small bets to provide the base level of comfort so that scientists and engineers can work with some piece of mind with the hope of having a few unicorn successes that make up for the rest. This seems like what's ostensibly happening in silicon valley to begin with so that's why I have this interpretation.
The implementation details always matter but my view is that the Kariko story is precisely in favor of the "fund people, not projects". Ultimately the article comes to the same conclusion ("... fund a more diverse range of people and ideas, even ... ideas that are currently ... unpopular, unworkable, obscure ...") but I guess the assumption is that a "fund people" approach would be implemented to favor superstars at the exclusion of everyone else.
The government funding doesn't necessarily have to identify the Karikó's of the world, as long as it can identify the Weissman's of the world who can identify The Karikó at the ad-hoc meeting.
This is one of the areas where "fund people not objects" really works. The people that are really productive are the first to know that they are heavily dependent on having a good team, and often those people are really good at finding hidden talent.
Karikó, for example, at her stage couldn't develop or attract a team if she wanted to.
I don't know Weissman so none of this is to comment on him. But I personally know many examples of this phenomenon (and have seen the Wizard behind the curtain when random events cause these cycles to get disrupted). It's that I think part of the problem the article is referring to is this kind of vicious circle and self-perpetuating funding and career cycles in certain areas of academic science.
I'm all for funding people rather than projects; I just think that it's only a fraction of the problem with research funding today.
With that honey trap in place, the economy will prosper outside and reward the Karikos.
This might actually work, but they are already famous professors in the fields. What to do with the newbies, who often have interesting ideas, but no previous body of work to back them up with?
1. Open access to research. Wikipedia, Sci-Hub, PLoS, Library Genesis, GitLab, MDPI, arXiv, PubMed, GitHub, BitTorrent, medRxiv, bioRxiv, and Tor. It's unconscionable that today professional societies like ACM and IEEE are using copyright to impede access to knowledge as if they were for-profit corporations.
2. Universal basic income, so today's Karikós don't have to choose between research and food. Conceivably some kind of reform is needed to prevent landlords from skimming off the UBI and returning us to zero, as the Georgists claim.
3. Reducing the cost of apparatus through initiatives like Foldscope and Paperfuge. We need not just one Manu Prakash but ten thousand Manu Prakashes, because Edmund Scientific is just never going to fulfill the needs of shoestring labs in Ghana.
4. Freedom from persecution for researchers. For example, where I live, buying acetone gets you Put On A List, and the hardware stores now label their lye simply as "drain opener" to evade the same regulations. California recently prohibited the sale of basic supplies like xylene under an extremely far-fetched interpretation of anti-air-pollution laws. Only recently did Texas repeal its prohibition on sales of lab glassware to unlicensed individuals. Hennig Brand discovered phosphorus by isolating it from a massive amount of his own urine; in South Carolina today you can get arrested for possessing a bottle of urine: https://www.avvo.com/legal-answers/a-friend-was-charged-with.... Critical Art Ensemble founder Steve Kurtz was famously arrested for "bioterrorism," then indicted for wire fraud and mail fraud, because he and his wife were culturing non-pathogenic bacteria in Petri dishes. This sort of thing should be unthinkable.
This will probably require ending drug prohibition; I don't see a reasonable way to protect researchers' freedom to synthesize arbitrary materials while prohibiting the possession of a large and constantly growing list of materials. But I don't think it's reasonable to prioritize the needs of drug addicts, drug traffickers, and empire-builders in law enforcement over the needs of innovators.
5. Going beyond freedom from active persecution, it's important to cultivate a social attitude that basic research is not only not harmful, but an important and worthwhile activity. This is the opposite of the attitude cultivated by Hollywood, which considers anything scientific to be inherently scary and antihuman.
Except for ponies. Those are pleasant but they're too resource-intensive to be much of a force for advancing basic research.
Seems like a really easy decision to make...
"I will save you five million dollars, if you give me three hundred dollars?". Just take my money and the damned horse is a gift.
But we can certainly aspire to liberate science from bibliometrics, dollar-auction postdoc rat races, grantwriting, mandatory reporting requirements, and especially prosecution, so that the people who do want to spend their time on advancing human knowledge have the opportunity to do so. And we can work to change the public perception of innovation and research from Frankenstein and Walter White to Edison, Tesla, and Einstein.
You can see what people do with total freedom by looking at retirees, the rich, the temporarily unemployed, people on disability, and NEETs. Some of them do the great things they aspire to. Most of them don't. I still think more freedom is a net good.
This is only one anecdote, but the only person I know which perfectly matches your typical child-beater profile, couldn't stand being unproductive on retirement and got back to work at local carpentry shop, as a helper. Not because he needs money, but because he needs something to do.
We need to have some incentive to at least share null-results and replications.
As a simple example, we've known how to make transparent glass for 1900 years, but it took Ben Krasnow two months to achieve it himself despite having money, libraries, and glass experts at his disposal, and being generally competent at making things, having built, for example, his own electron microscope: https://www.youtube.com/watch?v=mUcUy7SqdS0
I wish I had a panacea for these things...
You are not being truthful, labeling ivermectin only a "horse dewormer". Ivermectin is used as an antiparasitic drug also for people. And not only as a dewormer, but also against lice and mites.
I've taken ivermectin—for parasites, not for covid. I wonder how many people will die of parasitic infections or more toxic antihelminthics and acaricides in the next few years because they or their neighbors refused to take ivermectin after they heard it was a dangerous veterinary-only drug from unscrupulous liars during the covid pandemic.
Could it be that ivermectin helped third world patients with undetected worm infestation get rid of it and thus improved their covid-19 outcomes?
I guess 80% is good enough. Consider the advantages of focusing on your core theses, though.
They reformulated all the nail polish removers using ethyl acetate instead of acetone.
That said, the listings I'm finding on MercadoLibre right now don't have the requirement that you send them your DNI or your Sedronar authorization that I used to see, so maybe the policy changed recently?
Not saying that what you're referring to isn't also a problem, but the immediate problem with reference to the paper is the meritocracy-funding-complex that underlies modern biomedical academics (and by extension, other areas of academics that are held up to it as a profit source by universities).
Why not aggregate those independent researchers into a single place so that they can share ideas and the cost of high quality instruments? Like a university.
Also, I don't disagree with any of these ideas, I just don't think they're relevant in a university setting, at least in the US. Most universities have good libraries with access to all the luxury journals. They have core facilities with high quality equipment. There's some red-tape but you can study just about any chemical in a university lab.
Edit: I should note that the current journal publishing system is a terrible fucking scam and should be immediately dismantled. That said, it's not currently limiting my research because everyone at my university shares in the extortion fees charged by publishers.
I think it's likely that improving the options for non-institutional researchers will improve the situation for institutional researchers as well, among other things because it improves their BATNA in negotiations with institutional forces.
Probably none of "amateur scientist", "science hobbyist", or "citizen scientist" is good enough to serve as a reasonable reputational BATNA for professors considering leaving a university. "Independent researcher", maybe?
In the research literature I very rarely see references to "scientists"; that's mostly confined to the vulgar press. Instead it talks about "researchers", "workers", "investigators", or "authors", or until recently "philosophers". So from a shibboleth perspective "citizen scientist" is pretty bad; if you say "I'm a citizen scientist" you're implicitly contradicting your claim of membership by using the exonym.
And most women who get a Ph.D. sacrifice the chance to have kids in their 20s in the process, which is a big deal for many of them; life isn't easy for "nontraditional" postdocs.
And 95% of people don't live in the US.
Finally, universities aren't omnipotent. Gang Chen, David Gelernter, Aaron Swartz, Steve Kurtz, Star Simpson, and Majid Shahriari were all affiliated with universities, and they were persecuted for their research anyway, resulting in their deaths in two cases.
Also, many of the measures I suggest would substantially ease the path of researchers who want to form a new institution to pool resources, such as a hackerspace or a mail-order DNA synthesis company.
So, I think giving people more freedom to pursue research as they see fit would substantially increase the amount of research that gets done, and so would placing more social value on it.
https://maps.org/news/media/marijuana-moment-another-scienti...
For example, it is extremely easy, with both household and scientific chemicals/tools, to create a hazardous waste problem that is a problem for neighbors or a community. I imagine it could be the same for some biological experiments. One of the key advantages of garage experimentation is the freedom from the strictures of a laboratory, but one cannot lose sight of the impacts on others.
1. Create an environment without fear of repercussions for hazardous waste training and services. In much of the world today if you show up at a hazardous waste collection site with 50 kg of poorly characterized organic molecules dissolved in acetone, you can expect an investigation of whether you're running a meth lab.
2. Scale down the hazardous waste regulations to something realistic. A return to Victorian-era nonchalance about things like lead acetate and mercuric chloride would be counterproductive, but we've somehow gotten to a point where people call in hazardous waste teams if they break a mercury thermometer, which is totally out of proportion to the actual risk from metallic mercury, and I've literally seen a news report about sulfur spills where adult anchors who presumably went to college solemnly warn that sulfur is very toxic and can cause severe skin burns. Due to totally implausible concerns about toxicity, borax has been prohibited for most uses (fertilizer, flame retardancy, etc.) in the EU now, though it fortunately remains available in pharmacies. And I already mentioned California's restrictions on aromatic solvents on the absurd pretext of "air quality".
3. For chemistry in particular, focus on developing microfluidics and miniaturized apparatus, because a nanoliter of pyridine is much less of a nuisance than a liter.
but with relevance to both the Darwin Award to that science teacher that beheaded herself by throwing substances in a manhole is a warning about possible miscalculations, behaviours and events. https://darwinawards.com/darwin/darwin2008-06.html
How could we possibly design a system, whereby persistence alone is enough - enough for a liveable income, facilities to do research, and contact with fellow researchers who might help with the next piece of the puzzle?
Our concept of knowledge is ludicrously positivist - we tend to think we know where we are ignorant. I feel the reality is that we are surrounded by potential breakthroughs. The only problem is it might take your whole life - or longer - to make them work... and a specific one might not be there at all. So the answer is to let ten thousand flowers bloom, whereever they want (and not imprison them).
The odds are not great for individual flowers, so it's disingenuous to encourage them; but the odds are great, collectively.
Who is the better/safer bet to fund: Kariko or Weissman (MD PhD fellowship at the NIH under Anthony Fauci, full professor at UPenn)?
Perhaps it may be reasonable if someone misunderstands part of the research (if it did happen)
Meanwhile, these new institutes don't have to solve every possible problem with funding research to be worthwhile. They should be judged on whether they're an improvement to the state of the art, not on whether they solve this one problem the author cares about.
So in the end it is up to the researcher themselves to come up with the funds for their research by convincing someone with money. That can be the government, but doesn't have to be. Actually, it is probably NOT going to be the government, because they are just playing some sort of game in which they have no skin, and the money will go to people who play the game better. So it will be easier to find someone else who has some intuition about the problem, or about you, or both.
2. Safe proposals are generally rewarded, risky ones less so...but not based on scientific merit.
We know which institutions conduct science (i.e. accredited research institutions). NIH NSF should provide block grants to Universities, which would provide that to Departments, which would then fund the Department's scientists.
This would:
1. ...create more heterogeneously funded science because you Departments judge on personal interactions with individual scientists, who will have more intimate knowledge of their research and strengths.
2. ...risky science is more likely to be funded, and less politically adept science can get funded.
3. departmental/university applications for funding are much less work than traditional NIH NFS applications...because the University has a vested interest in not wasting their scientist's time.
For example, if we use "fund research at random", people will flood with research proposals.
Or if HN upvotes will become measure of which research is funded, soon enough people will optimize research proposals for HN upvotes.
I think, goodhart's law would not apply in certain situations, for example when signal is extremely costly to fake.
In research world, some unquestionable proof of personal time of a researcher spent would suffice, I think.
I.e three years in a row researcher comes in the same grant proposal and his colleagues say in writing, that yes, he is working only on it and nothing else for three years, no funny stuff.
The New York Marathon has a number of different selection criteria to handle different kinds of priorities: getting the world's fastest runners (qualifying marathon results), getting local runners (provide proof of residency), and raising money (let some people just buy their way in via auction).
Here's a nice podcast on their system: https://www.npr.org/2020/01/03/793488868/episode-962-advance...
But we could certainly imagine similar for research funding; different paths for rewarding known-good horses, innovative project ideas, etc.
I reckon they probably take some cut as a licensing fee, but seems like participating in the upside would create significant incentives over the current regimes.
Because if it was trivially easy to provide such "equity", the research would've been funded by non-government, private sources already. The whole point of public research funding is to deal with fields and scenarios where this is not really feasible.
The author is misguided if this is consider to be 'discussed in hushed tones as a cautionary tale for young scientists'. There are a myriad of applications for NIH funding and vast majority are rejected, repeatedly. It is unnecessary to embellish to make the next point.
> Yet she persisted.
This is indeed, the minority of 'non-tenure-track research assistant professor', most will get discouraged and drop out.
(&, 712 out of 2,005)
Reminder if you attended a uni in 2010 the tuition has now doubled to tripled on average, and the quality of education has vastly diminished. Silicon valley is funding labs themselves because uni's have become freeloading institutions we have to defund but are too scared to.
Imagine that Kariko actually dropped out of academia after unsuccessful postdoc, as most postdocs do. Does that mean we would never have mRNA vaccines? I don’t think so, I think that, at best, one could argue that we would have them later. How much later? The figure here most likely does not count in multiple decades. It is very typical for discoveries to be independently done by multiple researchers at roughly the same time. This is because many discoveries and inventions are made when “the time is ripe”, so to speak — when other discoveries and technologies set up the stage for the final leap. I do not mean to diminish the achievement of Kariko here in anyway — she was, after all, the first to actually do it in the real, non counter factual world. But, had it not been her, it would probably be someone else, somewhat later.
Now, one can argue that getting discoveries earlier is of crucial value. After all, we did hugely benefit from mRNA vaccine technology being available just in time for 2020. I course, I fully agree here, but again, it is necessary to consider counter factual. With some alternative modes of funding, we might be able to get Kariko to invent mRNA vaccines earlier, sure. However, in this counterfactual, we would also probably get a bunch of other discoveries later than we actually did. Which ones would those be? Are these more or less important than mRNA vaccines? It is, of course, impossible to know, and impossible to figure out before you actually decide to change your funding processes, because the process of discovery is fundamentally highly unpredictable.
Point here is that it really is not instructive to focus on a single anecdote when discussing making fundamental changes to the system, because the counterfactual world is way bigger and more complex than a single anecdote.
It seems more notable to me that two people working in the same university were unaware of their mutual interest in RNA. However, I have seen similar things in large companies. When I was analyzing R&D for a 1000 person company, there were several similar projects unaware of their mutual co-existence.
If your work gets funded and published, it's easier not harder for people working on tangentially-related stuff to hear about it and start cooperating with you. "Interdisciplinarity" is a big draw.
But anyway, the article states clearly that Kariko floundered for years until Drew Weissman working at the _privately_ funded University of Pennsylvania picked her up. You wouldn't have even needed to look up if the University of Pennsylvania was privately funded, because just about every important breakthrough in science in the last 30 years occurred at a privately funded American university.
There is a reason why something like 10 privately funded American universities make more scientific breakthroughs than the 10,000 publicly funded universities around the world put together. It's right in front of everyone's faces, and no one is talking about it. It is seriously and elephant in the room.
Then the article comes up with a "solution" like this
>My answer? Bend over backward to fund a more diverse range of people and ideas, even deliberately including ideas that are currently perceived as unpopular, unworkable, obscure, and the like. After all, many scientific discoveries can be traced back to origins that didn’t seem promising
Seriously, is this the best you can do? Regardless, this idea would be impossible in the first place at a government funded institution so why even bother mentioning it.
(UCSD, Berkely, UMich, and University of Utah, among others, are research powerhouses too).
Messenger bias?
this is just not true. They just hire the best of the best but if one deep-dives into any kind of research one can see an ocean full of universities producing valuable research. And most of the time the basic breakthroughs came from universities you didn't think of or you didn't even know.
This is simply untrue. When on deep-dives into any kind of research on sees an ocean full of universities polluting the sea with absolute crap. The more bureaucratic and government funded, the worse. For example, I know researchers who simply will not bother to read anything written by anyone working for a Chinese university.
But this:
> The more bureaucratic and government funded, the worse
is definitely untrue. You are thinking too much of hollywood-movies, this is not reality.
Just think of ML and Deep Learning. The university of toronto is publicly funded, as is the technical university of Munich, where schmidhuber (and hochreiter) produced a lot of the groundbreaking research we are all relying on. Or the university of montreal with joshua bengio. And the technical university of munich is not even in the states! SHOCKING! No private universities to be seen anywhere.
> just about every important breakthrough in science in the last 30 years occurred at a privately funded American university.
Really makes it difficult to take the rest of this comment seriously.
In case the ways it is "not even wrong" need to be detailed:
1. Drew Weissman's research was (almost certainly) majority funded by public money doled out by the NIH, NSF, and other national organizations. The public/private status of the university has little bearing on that, as most university research funding comes through these agencies, with something like 50% generally going directly to the universities themselves. Research at "private" universities as it currently exists would not survive without this mechanism.
> something like 10 privately funded American universities make more scientific breakthroughs than the 10,000 publicly funded universities around the world put together
2. Also very false, although arguably hard to prove one way or the other. How do you define breakthroughs? Press releases from university PR departments? Patents? Either way (or by some third—hopefully measurable—way that I'll allow you to define for us) I guarantee that you need to go much further down the list of private universities before you match the output of "all publicly funded universities around the world put together".
I imagine that you have in mind important (and/or well publicized) advancements from MIT/Stanford/Harvard and are forgetting about the enormous amount of research output from public universities (which include but are not limited to Berkeley, CalTech, U of Michigan, Georgia Tech, Virginia Tech, U of Texas, Ohio State, etc.)
> Regardless, this idea would be impossible in the first place at a government funded institution so why even bother mentioning it.
3. As you hardly cite any evidence for this, I'll point out that private money can go to diverse people at diverse institutions (including government funded universities).
So my question to _you_ is whether this comment is motivated by a knee-jerk anti-government reaction, or if I'm entirely misunderstanding where you got these ideas?
At least where I live, many government grants are only available to people who have also managed to get private industry funding for their work too. These grants are usually very successful. This does not disprove my point in any way, in fact, it _is_ my point
>2. Also very false, although arguably hard to prove one way or the other.
You have to understand that it is in the interest of the tens of thousands of people doing work doing non-sense research to pretend their research is important. Just because you hear about them telling you how important their worki is in the media, doesn't mean it is.
Anyone who has actually worked it research knows every field is filled with 10's of thousands of garbage research papers that are of no value, and that all the key work is produced by just a hand full of people. I remember also reading some researchers that looked at dozens of fields and breakthroughs and found the same thing. All the real work in any breakthrough is done by just 2 or 3 people at most. so this is incorrect, what I said is actually very provable.
>I'll point out that private money can go to diverse people at diverse institutions (including government funded universities).
Yes, that is my point...? You are calling my comment a knee jerk reaction yet you have responded without seeming to understand any of it.
In my experiences, the current system has led to problems with a small number of people "sucking up credit" that's unwarranted, in the sense that they're very very good at taking credit from others and building up a CV that makes it look like they're at the center of things.
In any event, I'm very skeptical of these things at this point based on my personal experiences. Usually progress is incremental and involves a lot of efforts from lots of individuals. Even bigger advances usually involve a confluence of things.
Bibliometric studies are often flawed because they make a lot of false assumptions and ignore realistic dynamics, with corruption and gaming of metrics.
Grants like that are rare, because there is very little industry funding for basic research. Private funding usually comes from various trusts and foundations that operate in similar ways to government funding agencies.
> I remember also reading some researchers that looked at dozens of fields and breakthroughs and found the same thing. All the real work in any breakthrough is done by just 2 or 3 people at most.
Alexander the Great didn't win battles on his own. He needed a lot of soldiers for that. Similarly, scientific breakthroughs are meaningless on their own. You need a massive amount of grunt work by ordinary researchers to connect them to the real world and make them useful.
> >1. Drew Weissman's research was (almost certainly) majority funded [by public money]...
> At least where I live, many government grants are only available to people who have also managed to get private industry funding for their work too.
I don't believe this is true at Penn, or other universities that I'm aware of in the US. I don't doubt that Dr. Weissman is able to get private foundation funding or corporate sponsorship, but that's very likely an effect of his private funding, rather than the other way around.
> > > something like 10 privately funded American universities make more scientific breakthroughs than the 10,000 publicly funded universities around the world put together
> >2. Also very false, although arguably hard to prove one way or the other. [... How do you define breakthroughs? Press releases from university PR departments?]
> You have to understand that it is in the interest of the tens of thousands of people doing work doing non-sense research to pretend their research is important. Just because you hear about them telling you how important their worki is in the media, doesn't mean it is.
Yes, this is basically my point. Just because you can bring to mind "breakthroughs" from major private universities, doesn't mean that most valuable research is conducted there.
> Anyone who has actually worked it research knows every field is filled with 10's of thousands of garbage research papers that are of no value, and that all the key work is produced by just a hand full of people.
As someone who works in research I agree that there are many garbage papers. I absolutely do not agree that the key work is done by a handful of people...unless hundreds or thousands of productive researchers (depending on the field) counts as a handful.
> I remember also reading some researchers that looked at dozens of fields and breakthroughs and found the same thing. All the real work in any breakthrough is done by just 2 or 3 people at most. so this is incorrect, what I said is actually very provable.
I still don't know how you're defining breakthroughs. I suspect that you are referring to how much work any of the (let's say 10) authors on a single paper did in that publication. Yeah, 1-3 people on any given paper sounds about right. But most/all of those (7-9) other folks are the key people on other publication. This does not mean that most researchers are not contributing to science (although how we allocate credit in authorship could certainly use some fixing).
> > > Regardless, this idea would be impossible in the first place at a government funded institution so why even bother mentioning it.
> > I'll point out that private money can go to diverse people at diverse institutions (including government funded universities).
> Yes, that is my point...? You are calling my comment a knee jerk reaction yet you have responded without seeming to understand any of it.
I read this part of your comment (and, in fact, all of it) as saying that public universities are inferior because people working there cannot pursue ideas that the _government_ deems unacceptable because they are government funded. Throughout my response, I'm pointing out that the public/private status of the university has little bearing on where researchers' funding comes from. Not only do most researchers at private universities get public funding, but there's nothing stopping researchers at public universities from receiving private funding.
I believe that you have a fundamentally incorrect understanding of how research and research funding (in the US) works.
1. It requires quite a large investment which disfavors smaller institutions
2. The distribution of patents that actually make money is so incredibly skewed that university patents should considered such a bad investment, that it wouldn't be a good metric anymore.
There's certainly a lot of fad-chasing in biomedicine. There are strong incentives to flock to the latest technique (RNAi, optogenetics, single-cell-seq, organoids) or stick close to an established hypothesis (beta-amyloids for Alzheimer). It might better if there were some countervailing incentives to stick with older techniques, or break away from established dogma. For example, optogenetics has been a very powerful tool for understanding the brain, but it also means that we've moved away from many animal models in favor of mice, where the tools are most tractable. As a result, we know more about brains that are, in some ways, less relevant to human health. A-beta, on the other hand, has been an absolute tire-fire and the field should have shifted ages ago.
The NIH tends not to 'steer' the field in particular directions, but they could. DARPA programs, for example, sometimes explicitly fund several competing ideas to see if a clear winner emerges.
On top of all that, we should also be building a workforce of diverse researchers.
e.g., the government will match any private money 2 to 1. Then, for example, the government can take a cut of patents that result (to prevent grants simply subsidizing research that would have occurred anyway).
Anyway, mechanisms like what you're proposing do exist (MITACS in Canada, for example). However, the problem is the other end of the pipeline. How can you pitch investors on something that might not be a product for decades, if at all? Government R&D primes that pump.
You are mistaking privately owned universities for privately funded. Scientists in privately owned American universities still mostly apply and receive funding from public (federally owned) funding agencies.
In America these privately owned universities use their own money to hire the best people. Private money does an infinitely better job of selecting the best people than government money.
Then the government notes the historical success of these private institutions and sends funding their way, which the excellent individuals they chose can use to create excellent research. This system has made America produce orders of magnitude better science than the rest of the world put together.
My point is that the question of how the government can best direct money has already been solved. Let the private sphere decide who is the best.