Hacker News Reader

Comment by ThePhysicist | Hacker News Reader

Science in general needs more staff scientists. Currently the system is based mostly on "cheap" labor by students and postdocs who often work long hours for minimal pay in the hope of one day getting a real position. This is not only demotivating and unfair but also very inefficient, since labs will often tend to solve problems using manual labor instead of seeking more innovative solutions.

Personally, I think a well-run lab needs a combination of staff scientists, very good engineers and salespeople, with one or more visionaries at the top who define the direction of work. OpenAI is a great example of how this can work (IMHO),as they have both excellent scientists but also brilliant engineers and people that are very good at selling their vision to the world.

View on HN

>Science in general needs more staff scientists

Science needs to be better funded with fewer strings attached to dollars. How much of the abilities of our best scientific minds goes to writing grant proposals?

Science is very well funded in the U.S. Requiring selling your ideas is important to put funding where the good ideas are.

Some are better at selling than others though.

It's a competitive market as it should be. To weed out the ideas that shouldn't be funded.

A better way to very ideas?

Applied science surely needs to rest on a healthy bedrock of pure science to draw ideas from. I don't think you can advance science purely by relying on what might have "real world" applications. It's a process of discovery.

Also remember lots of "useless"/theoretical science turned out to have enormous real world uses.

My comment said nothing about "applied science" vs. "basic science." I'm actually talking about basic science.

You can do basic science but have very bad ideas based on very poor data which you interpreted poorly. You shouldn't be funded. There are enough people with good ideas for basic science who should rather get the money.

This misses the point that grant agencies are rubbish at funding great ideas. And also that some scientists shamelessly oversell their ideas and even falsify data - behaviour which is of course encouraged by the system of perverse incentives you advocate.

Yes, but they're at least adequate at funding good science. That is all that they are attempting.

I did mention that there are people with salesmanship and that's an advantage that you really can't control for. Well except through scrutiny of only the science, which seems to be fairly well done through a group of scientists who have to approve proposals.

"Oversell ideas and falsify data": Of course a competitive market will encourage these behaviors. But just because there are unethical people in science doesn't mean that we should do away with the system.

I would add to "falsify data", misinterpret and misunderstand data (and specifically the statistics involved). Again, something you can control for to some degree with scrutiny of the applications by other good scientists.

I don't know about adequate - there are lamentations about science funding from young and old researchers alike, all the time. Many breakthrough areas struggled along for years with little funding, while a ton of money went into dead ends. Eg immunotherapy in cancer, H. pylori as the major cause of gastric ulcers...

Any system you implement creates incentives, these need to be properly aligned to ensure an effective and humane system. If your system encourages or rewards unethical behaviour that is a big problem.

Having applied for and reviewed grants, the idea of 'good scientists' scrutinising a proposal carefully and controlling for all these factors is a bit fantastical. Maybe in your area it's different.

> Many breakthrough areas struggled along for years with little funding, while a ton of money went into dead ends. Eg immunotherapy in cancer, H. pylori as the major cause of gastric ulcers...

This looks an awful lot like survivor bias.

Tons of money was also put into well established research to move it along. And ideas that were stupid also didn't get any funding and struggled.

> If your system encourages or rewards unethical behaviour that is a big problem.

That's markets and that's why we have a justice system. A reason why the NIH now requires grant applications to be of publishable quality to help diminish completely turnaround in experimental design after grant have been issued. A part of the evolution that I speak of.

> the idea of 'good scientists' scrutinising a proposal carefully and controlling for all these factors is a bit fantastical.

I doubt you want to do away with peer review. I'll be the first to say that It's not perfect, but it's gotten us pretty far.

"Good scientists" have their orthodoxy and "know" certain things to be to true and won't (most of the time) fund anything that goes against that orthodoxy. I.e. one funeral at a time.

Well funded PIs have the ability to fund this type of science and many do.

I think we're pretty much in agreement though.

The system is slow, not great, but like another commenter suggested, the best we can do.

> This looks an awful lot like survivor bias.

I'm not sure it's survivorship bias.

H. pylori, for example, actually is a major cause of gastric ulcers. If we were actually good at identifying good ideas, Marshall and Warren would have had a pretty steady rise to fame over the course of their careers since they had, in fact, figured it out. For what I've heard, they did not--they were treated pretty shabbily for a while, then the field did an about-face and they won a Nobel prize. It's not the case that anyone who (somehow) survived that long gets a Nobel!

> I doubt you want to do away with peer review. I don't think anyone wants to completely do away with the idea of having others sanity-check their research. However, there is a huge gap between peer review as an abstract concept and peer review as currently practiced in NIH study sections. Personally, I think that there's a lot of room for improvement in the latter. For example, you could imagine giving program managers more room to actually manage a portfolio of research and (e.g.) hedge across multiple hypotheses.

> I'm not sure it's survivorship bias.

I'm referring to good ideas being underfunded for a time. Only good ideas survive. Bad ideas obviously don't get to a point where they turn into something accepted and widely applicable (I'm sure there's a countercase or two...)

So only the good survive. Does that mean we need to fund across the board? That's just not feasible, so we have some good ideas that go unfunded, but a whole lot of bad ideas that do not.

> manage a portfolio of research and (e.g.) hedge across multiple hypotheses.

I see this happening in well funded labs. Taking a bit of money from here and some from there in things that are novel, but ethically related enough to be able to be justified.

But the ethics involved makes this dangerous.

You bring up good points.

This reminds of that Churchill quote about democracy: "the worst form of government, except for all those other forms that have been tried." Besides competitive grant awards, do you have a better mechanism for reliably funding brilliant ideas? I doubt it. I agree the system could be improved in certain ways, like tilting the funding stream less heavily towards older PIs, and funding a few moonshots, both problems the NIH at least claims to be addressing now.

Also, some scientists do overhype their ideas, and a very few might make up data, but the majority do neither.

The problem is of course your argument justifies any incumbent system no matter how heinous. I don't think the analogy holds up. And I don't think I am required to present a plan to revolutionise research funding in a comment on HN.

If the current system is so "heinous" then seemingly that should not be very difficult.

I don't think you're required to no, but you can at least attempt an answer or cite someone else that has put thought into an alternative.

And just because you fail at your attempt does not make your initial point void.

The idea that science funding at this point is a function of "What ideas are good" is pretty contrary to actual evidence.

Care to elaborate?

To get funding you have to provide a hypothesis and aims to prove/disprove with a starting justification for said hypothesis.

If your hypothesis is good compared to someone who has a poor hypothesis you should get the funding. We can't just open the funding floodgates to everyone. Not all people with graduate degrees are good at science. And only the best should be rewarded with funding.

Again, there is unfortunately an advantage if you're a good writer, but I don't think there's a better system at this point in time that we can rely on except maybe past published work.

What you described is what should happen.

What actually happens is you provide a research question and the way you intend to approach it.

What determines whether you get funding is a complex function of your age (don't be young), gender (don't be a woman), race (don't be non-white), history (come from a big lab with a prominent PI), whether you've been funded before, weird reviewer preference (like, seriously, whether you put citations in your specific aims), what section you went to and whether or not the review panel thinks this is worth doing, whether you lie in the comfortable space where your work is neither very safe nor particularly risky, etc.

Plus stochasticity.

We might not be able to "open the funding floodgates" to everyone, but at the moment it's hardly a meritocracy, and it's strangling junior scientists. It's been fascinating to see things evolve even during the relatively short period of my career.

> What actually happens is you provide a research question and the way you intend to approach it.

Which is what I described. Except that I'm comparing two identical candidates except that the one has better science than the other, which is the genesis of my thesis: Bad science should not get funded.

All of the things you mention come into play at one degree or another depending on field. I think with some experience can agree to this, but even with this, I don't see how an alternative approach would make it better than what we currently have.

Evolution is really what's happening, and from my perspective for the better.

> Age

Advancing one funeral at a time...

> Gender.

The effect of this is field specific. Biology for instance skews female and I've met some fiercely competitive female biologists who are very successful, but besides my anecdote, females are preferred in biology overall.

> Race.

My sample size is a little small but does not skew Caucasian.

> History.

Which I don't see as a bad thing. Except if a PI makes the mistake of hiring an incompetent (but very aggressive) postdoc that can't tell a nude mouse from a barbering one... I've seen this happen and yes, the person carries clout from the lab with them.

> Funded before.

I don't see this as a bad thing either. As there as numerous (though not nearly as large) sources of funding only for young scientists to help alleviate this issue.

But this + history = track record. So bad for some, but good if controlled for bad science. But this too can be overcome through aggression and politicking.

> Reviewer preference.

From my experience a lot of established scientists outsource this to younger scientists so that they can get experience judging good science.

But yes, this is all over the place. But for all the shortcomings like the randomness you mention, this is a good tool especially if the reviewer pool is somewhat diverse, but also experts, a hard combination for sure.

> Worth doing

Or competes with things they're working on in secret...

> it's hardly a meritocracy

You meant to say Utopia right?

If you don't expect it to be a meritocracy then you'll be happier in the long run. As long as the system is evolving in a good direction, which from my perspective it is, we can ignore that it is not a perfect meritocracy. But creating perfect markets is pretty hard to do in general.

> Advancing one funeral at a time...

This is not a good thing.

> The effect of this is field specific. Biology for instance skews female and I've met some fiercely competitive female biologists who are very successful, but besides my anecdote, females are preferred in biology overall.

Biology skews female. Success in biology grant applications does not. https://www.statnews.com/2016/07/29/women-in-science/

> My sample size is a little small but does not skew Caucasian.

NIH funding does. https://www.nih.gov/news-events/news-releases/nih-commission...

> Which I don't see as a bad thing. Except if a PI makes the mistake of hiring an incompetent (but very aggressive) postdoc that can't tell a nude mouse from a barbering one... I've seen this happen and yes, the person carries clout from the lab with them.

But this entirely undermines your concept of this being a meritocracy. The name behind your former job doesn't mean your idea is any good, but we'll give you the benefit of the doubt.

> I don't see this as a bad thing either. As there as numerous (though not nearly as large) sources of funding only for young scientists to help alleviate this issue.

They are manifestly failing.

> But this + history = track record. So bad for some, but good if controlled for bad science. But this too can be overcome through aggression and politicking.

This isn't necessarily good for science. http://science.sciencemag.org/content/354/6312/aaf5239

> From my experience a lot of established scientists outsource this to younger scientists so that they can get experience judging good science.

Not on study panels they don't.

> If you don't expect it to be a meritocracy then you'll be happier in the long run. As long as the system is evolving in a good direction, which from my perspective it is, we can ignore that it is not a perfect meritocracy. But creating perfect markets is pretty hard to do in general.

It's not evolving in a good direction. Science funding is accumulating in a smaller number of "long tail" well funded labs. Science is hemorrhaging young investigators, and especially women and PoCs. The average age of a first major independent award is climbing. Paylines are going down. https://nexus.od.nih.gov/all/2017/03/06/mid-career-investiga...

I really can't see how you can possibly look at this and go "Oh yeah, this is going good places."

(this is getting a little long, feel free to focus on one or two items if you decide to respond.)

> Advancing one funeral at a time... > This is not a good thing.

That's my point, but it's not as big of a problem as blog posts make it out to be. Not small by any means though.

> Biology skews female. Success in biology grant applications does not.

I'd like to see proof of this. Let me quote some dubious lines.

> "women are underrepresented in high-ranking positions."

This is to be studied, but the statement in itself means little. Without analysis we could compare with fields where women are over represented. Analysis required.

> "The exact reasons for this attrition are debated"

Attrition? As far as I'm concerned the rates of females in higher positions, while low, has not gone down. If we mean women in science in general then that also has to be analyzed.

> “People are more likely to study issues relevant to their own health,” she said.

Nonsense. People study where there's funding. Breast cancer is studied more than prostate cancer? Give me a break.

Is there no bias? No. What affect does it currently have in funding. Your article points out, we can't tell.

> "The discordance between written feedback and scores suggests subconscious gender bias bleeds into the review process — influencing both male and female reviewers."

So because we tend to praise women more than men and accept failure in women more than men that means what exactly? We should also fund them more easily than men?

> “These findings confirm that it influences us.”

Yes, but this doesn't, by itself, mean anything negative. Which the paper admits.

> "Subconscious gender bias may lead reviewers to hold women to higher standards than men"

It doesn't seem to be possible to get this from the data. I can claim that society is much more tolerant of failure in women but I can't claim this purely from the data in this research, the converse is also true.

> "The authors' analyses suggest that subtle gender bias may continue"

The authors of the study can't even make the claim that you want them to make.

Back to your claim:

> "Success in biology grant applications does not."

The study claimed nothing about success.

> "NIH funding does." ... skew Caucasian.

Not if you're American. Read the link you left here.

I would personally highly agree with this line from your link: "While further examination is needed to identify causes for the differences in NIH R01 grant success, one possibility the authors cite is that the quality of educational and mentoring experience"

But that gets political.

If we're talking about immigrants then again African and West Indies education is sub par (I'm African.) which is a major confounding factor.

> The name behind your former job doesn't mean your idea is any good, but we'll give you the benefit of the doubt.

No, but it means that you likely got better training than someone from an underfunded non-competitive environment. This is one way to judge (even though it is not a perfect test) someone's likelihood of being a good scientist. I think you would agree that good training plays a monumental role in developing a good scientist.

> They are manifestly failing. (Funding for young scientists.)

How so?

> Not on study panels they don't.

Not really relevant to what I was saying about reviews.

> This isn't necessarily good for science. http://science.sciencemag.org/content/354/6312/aaf5239 (in response to track record being important).

I didn't claim it was perfect. Nor did I claim that it will lead to discovering and funding scientists that will have great impact and letting those sometimes fall through the cracks.

What would better? If random selection is better then we can start having an argument. Not that we could really test this.

> Science is hemorrhaging young investigators

* mid career.

And this should follow with a question (that isn't answered by the research you cite): why?

> , and especially women and

Do we have any proof of this? If you were to say "attract" instead of "hemorrhage" I may more easily agree. But in some fields which females seem to prefer this is overwhelmingly false, see health.

> PoCs.

This does not seem to be the fault of the funding institutions.

> I really can't see how you can possibly look at this and go "Oh yeah, this is going good places."

Because institutions like the NIH are making evolutionary strides at allocating money to good science, slowly removing the benefit of unethical behaviour among other things.

Most of the problems you cite though are not related to the funding institutions but to externalities.

Bias, education, governmental funding etc.

>Science is very well funded in the U.S.

Then why do the grant agencies have a constantly-declining acceptance rate, and budgets that basically stay constant every year in real dollars while the economy as a whole grows?

I didn't say that it was growing at some arbitrary rate.

Something being well funded does not mean that that funding will improve over time.

If R&D is 2% of GDP and declines slightly over time but remains in the top 10 we can still call it very well funded.

And just because acceptance rate is declining and competition is increasing still does not mean that science is not very well funded in the U.S.

Exact... in the U.S.

I meant to type "A better way to vet ideas?"

I completely agree with this. I believe that staff scientists should be treated as independent researchers. In my experience, they tend to be used as lab support, for example, ordering reagents, managing the annoying aspects of big projects, etc. etc.

As a post-doctoral fellow who would rather not be a principle investigator, I would love a job where I can work independently on my own research projects with a reasonable wage.

> In my experience, they tend to be used as lab support, for example, ordering reagents, managing the annoying aspects of big projects, etc. etc.

The thing is, as a PI, someone has to do this. And while I too would like to spend my time working independently on research projects and not having to do time consuming lab support stuff I'd rather not do, I also need to be writing grants to keep my staff scientists paid.

Also, the whole argument to staff scientists representing institutional knowledge also applies to support stuff. "We don't order from X any more for reason Y." "Collaborators at Other University are usually two months late, built that into the timeline", etc.

You could try to get into the French system. They have more staff positions (permanent contract based) but of course at a lower salary.

Federal research (i.e. civil servants at NASA, NOAA, DOE, etc.) tends to work closer to that model, but not 100%.

Twenty years ago when I worked for a FFRDC (Mitre), the number of staff were preallocated for the entire year, and the only decisions were how to allocate personnel onto the pool of candidate projects. As long as you could interest a gov't project manager, you got approval to go ahead with your (their) work. However, our capital expenditures were generally pretty minimal. This model allocated labor almost exclusively.

That's about as liberal an R&D policy as I've seen anywhere.

>This is not only demotivating and unfair but also very inefficient, since companies will often tend to solve problems using manual labor instead of seeking more innovative solutions.

FTFY, since I think you just described the problem with the low-wage economy as a whole. But it's especially bad in science, where innovation and knowledge are the core competency.

such a lab setup needs huge money just for salaries, it is not possible for most lab in current funding environment.

Reply on news.ycombinator.com