Comment by opportune | Hacker News Reader

Comment by opportune | Hacker News Reader

Parent Full thread

A lot of the points you bring up re related to cost. Here's the thing about cost, let's say it costs $100k/year to hire a good software engineer capable of writing scientific code (able to program and test complex algorithms, write HPC code, turn whitepapers into code), which might even be an underestimate depending on how benefits are paid out and the area. You can also fund 3 more grad students for that kind of money. The grad students will directly convert a PI's money into authorships while the software engineer's contribution will be only indirect, and likely take years to pay off.

Plus, with only a single software engineer, there's a good chance you get unlucky and end up with someone clueless/lazy. You would probably need 3-4 software engineers to make a functioning team with best practices and hedge your bets against accidentally hiring someone who sucks. So now we're talking 10+ grad students.

Open source software is a bit different because many labs can band together to fund things they find useful. But again there are still issues with cost-effectiveness. I'm guessing most lab contributors to OSS would want some sort of quid-pro-quo which may not be realistic for all OSS projects. And by funding OSS you are also funding competing labs' abilities to use the same features you use, which is good for science in general but not good for people's careers sometimes

You're right, and more generally all of the issues are related to the fact that scientific incentives don't typically align with good development. At the end of the day, over a period of 3+ years, I'd rather have the results of 3-4 software engineers compared to 10-12 grad students. However, for <3 years, I'd choose the grad students. Pretty much every incentive in science (e.g. grants, awards) prioritizes being prolific over a short period of time.

Here in Oxbridge there's pretty generous funding for hiring software engineers to maintain some biotech software, and much lower pressure to publish. I know many open positions. And they tend to fund you for very long time. I know people working on said positions for more than a decade. Sometimes 20 years.

The problem is more pay gap with industry. Even though they tend to pay well by academic standards (e.g. no PhD required, yet pay is much higher than a senior postdoc), the salary is still way below what industry in London or Oxbridge offers.

Furthermore, you tend to be surrounded by non-technical people which may be tough in the long term. Nobody appreciates what you do. Not even your boss, who may know zero about computers.

The bottom line is that positions end up being vacant for long time and tend to be filled by biologists with a bit of coding experience, underqualified IT people or, rarely, really competent individuals that want a break from industry.

> The problem is more pay gap with industry. Even though they tend to pay well by academic standards (e.g. no PhD required, yet pay is much higher than a senior postdoc), the salary is still way below what industry in London or Oxbridge offers.

I know sufficiently many good programmers who would love to do scientific programming (because they love science) and would immediately accept less pay.

The problem, in my opinion, rather lies in the non-monetary work conditions. For example, in Germany it is nearly impossible to get permanent employment contract when working at a scientific institute and doing something remotely related to scientific work. Even worse: you are not even allowed to work more than 6+6 years (before and after doctorate) in a fixed-term employment position at a scientific institute If this time is over, you are not even allowed to take any non-permanent contract at a scientific institute (the infamous/insane Wissenschaftszeitvertragsgesetz (WissZeitVG)).

No programmer (even if (s)he has a great passion for science) will be willing to work under such extraordinarily bad conditions.

That sounds bad indeed.

Here it is a bit better. Most positions I know of are de facto permanent. On paper they are not, as most labs go through 5-year funding cycles. So there's a tiny chance of loosing funding. It's quite rare for big labs.

Besides, places like MRC have created permanent research assistant positions. Which are actually permanent and put zero pressure on publications.

As you say, connecting with interested and talented programmers is another problem. I feel that Nature Jobs postings, which were already a big leap forward for rusty uni administrators, are not good enough.

Why hire an engineer? Why not walk over to the CS (might have a different name) and talk to a professor there. They can set you up with plenty of undergrads who need this experience, and they should be able to guide them into something that is maintainable long term.

Note that I said should there. How to write maintainable programs seems to be lacking in research area.

Because undergrads are on average very bad programmers. They're both enthusiastic and don't know what they don't know. Good programmers are above all very experienced and disciplined and clear thinkers. Undergrads are stressed by their remaining workload so they have split responsibilities, and they just can't put the necessary time in.

I was such a CS undergrad working in a lab once upon a time. I don't think you really want that because the undergrad will probably only be working for like 4-15hr/week potentially for only a single semester. For a summer position, sure it's 40hr/week but still only for about 10-15 weeks.

And still, you're getting a generic CS undergrad's caliber of work and responsibility which I would say on average is not great. They might not be as familiar with version control, best practices, etc. and could just end up writing code just as bad as the scientists.

I think if hiring a team you would need at least one somewhat experienced full-time software engineer to act as team lead/PM for the other developers, whether fulltime or students.

Yeah, picking up undergrads (or even grad students) from the CS department is not a surefire way to end up code that follows best practices, is well maintained/documented, etc. and I'd probably argue if your team leader doesn't have that experience, you're more likely to end up with something that doesn't follow great practices (especially with regards to any sort of test suite).

All the replys about undergrad quality are correct. I stand by my statement though: we need to figure out how to solve this problem and research is sorely lacking.

This would be a good solution, but from what I've seen with psychologists and statisticians, it's unlikely to happen for reasons I don't fully understand. Another thing is that undergraduates often learn by adopting the norms of the institution that they're in (e.g. using version control, linters, etc) but when they're brought in as the technical person, they don't have that opportunity to improve (This is my personal anecdote as that cs undergrad at one time).

I’ve been that CS undergrad in the past who aided another department’s research, and the quality of my work back then was every bit as bad as you might expect. Of course I, like many of my CS peers, arrogantly thought I was doing great work at the time...

For sure, talented undergrad CS developers exist, but in my experience there are far fewer of them than many people might think. Experience counts for a lot, especially when trying to deliver even average quality software work.

As for something made by undergrads that is “maintainable” to a standard broadly comparable with an experienced developer? Maybe if you get lucky...

> plenty of undergrads who need this experience

The fact that they need the experience should already tell you that they're not yet competent at what they are needed for. They should be put on low-impact projects that don't matter, or low-impact projects that do matter with solid mentoring. They should not be made to write software for something that is high-impact for you with little to no mentoring.

Incidentally, this is exactly what my lab does

Reply on news.ycombinator.com