The pressure on the professor and the selection for obsessiveness explain motives.
Collectively, professors are the ones who set almost every aspect of the culture in academia. They are the ones who populate committees that set all the rules, and decide when a professor is being abusive and when he/she isn't. Professors decide how much is "enough" for a PhD. Even the pressure on professors comes mostly from other professors. A lot of people in the funding agency's committees are professors or former professors. If any change is to occur, it has to come from professors. No external or internal group really has any say in the matter.
So as a professor, the burden of change is pretty much on you. It's probably risky for you to do anything about it[1], but no one else can. When an outsider looks at the situation, all they see are professors pointing fingers at other professors as the cause. It is your profession to fix. And you have less to fear than most workers in most industries: Once you have tenure, attempting to fix the problem will not cost you your job. It will cost other things, but that's the point where it becomes clear what a professor's values are.
And you kind of skirt around it, but a big aspect of it these days is essentially the "rite of passage". As an example, I had a group mate who continually cursed his advisor because he wasn't letting him graduate and was being given work unrelated to his thesis just so that the professor could squeeze as much out of him as possible. Yet when he graduated, he said "Of course I'll treat my students the same. If I had to go through all this, then so should they!"
That's not an uncommon sentiment amongst professors.
Let's all keep in mind: This is mostly a US problem.[2] I don't normally hear these complaints in Europe, and most students there get their PhD in 3 years after their MS.
[1] Not really - there are lots of small things an established professor can do that help.
[2] Well, OK. Maybe also a Korean problem.
But outside of this context, these same explanations are exactly the excuses that professors and administrators will ultimately make.
Don’t get me wrong, industry isn’t perfect either. But I know which one I’d rather pick, and it sure as hell isnt academia.
Moral of the story: there's always someone willing to sacrifice more than you whether its their health, money, life, ethics, whatever.
If it were up to me, I'd convert some MS/PhD slots into staff scientist roles with longer contracts. I think you could probably do this in a way that increases productivity, and makes more people more happy to boot.
If often wondered why labs don't hire regular developers to increase research veliocity. I have no interest in research, but would happily work in the context of academia doing thing like handling merges, ensuring code modularity, maintaining infrastructure, writing unit tests, etc.
A friend of mine worked at Hopkins for a few years, did interesting stuff, and walked out with a (free) MS afterwards.
I visited APL a while back and loved it. The job didn’t work out (federal stuff has been....turbulent lately), but hopefully it’ll work out one day.
Thanks for the pointers!
Part of the problem is funding. You can get 3-4 grad students or ~2 postdocs for the price of a developer. Plus there are lots of existing mechanisms for funding them: training grants, internal and external fellowships, working as a TA, etc. A developer would have to get paid out of research funding, which is already pretty limited. The National Cancer Institute had a program for staff scientists, and the Chan-Zuckerberg Initiative just launched one targeting microscopy, but there aren’t tons of options, especially not for open-ended roles.
There’s been some adverse selection too. I briefly had a programmer but it rapidly became obvious he was working for an academic salary because no one else in their right m8ns would pay him more. I ended up rewriting all of that code, and despite this, my boss keeps sending me fresh-faced undergrads “to do the coding.” I guess the idea that you get what you pay for hasn’t sunk in yet. That said, we’ve also had a few that were excellent and were interested in the projects; I think they were both hired as part of some complicated arrangement where their spouses were recruited for more traditional academic roles.
Another part of it is that code quality hasn’t been a huge priority. That is finally starting to change, but most labs have a lot of code that was unceremoniously promoted from “one-off prototype” to “critical infrastructure” without too many changes. (This, incidentally, gives the lie to peoples’ obsession with YAGNI).
Finally, if anyone does need a neuro/ML themed developer, I call dibs :-) Seriously though, I completely agree that we have have more specialized roles (dev, technical writer/editor) and I think that whatever place manages to make this work could become a research powerhouse. Some of the bigger institutes (the Broad, Janlia Farms, etc) do have some jobs like this already.
And academic research often just needs to be “good enough” for that next paper or grant. Investing lots of time and people into code quality isn’t worthwhile unlesss your whole goal of the lab is to provide software as your output (there are a few, like the Wikipathways group). Bht for everyone else, the ROI is too low, better off working on the next grant or manuscript.
The catch is that it needs to both evolve and stay solid at the same time: it’s hard to predict what you’re going to want in three years—-or find the time to clean up code from the last three years, especially since many of those folks will have moved on.
My claim is that people systematically underestimate the value of good code to a research program. Good infrastructure lets the lab focus on the scientific questions, rather than the logistics of moving and processing the data, which in turn allows them to publish more, better, and faster. This is true for a lot of things: some labs have fantastically good imaging pipelines, or have worked out how to rapidly train animals for certain behaviors, or can reliably do an assay that often fails in others’ hands, and derive a huge benefit from it. Some are so good that I wouldn’t even consider competing with them in their niche. My argument is that good code can also have returns like that.
As a personal example, my first paper at McGill took about three years to finish. The next took about a year and a half (and just came out). We’re on track to submit at least one—-and maybe as many as three—-papers this year. Some of this is due to practice, but a lot of it is due to the fact that we built reusable components instead of “the script that gives the numbers”
I have a Masters, and all the interesting jobs I want to do are held by PhDs (sometimes with a couple years of postdocs). And everyone who's on that team has a PhD and they're definitely not going to let anyone lesser than that onto their team.
They all seem to get along fine, and the ones who have PhDs are usually playing down their credentials.