Ask HN: Why aren't more programmers in academia?

Academia requires more than being just a hacker/programmer and the money is not as good.

I share your experience with hacking in academia. It's amazing how much almost every scientific field has demand for academic programmers. Its not restricted to computer science. There is computational- subfield for everything: computational biology, chemistry, physics, brain science, economics. Almost all research requires some programming and tool development these days.

There exist similar positions in industry research labs with better pay. If somebody really wants to spend most of their career as a hacker–and not just writing bean counting apps for business–getting degree in some other field and then finding a job where they can program a lot is a good choice. Petrochemical research engineer who is a good programmer and knows computer science can have really fun problems to work with.

Agreed. One of the most interesting and challenging projects I worked on was a stratigraphic suite of programs used by a mining company. The programs had been written over a period of about 15-18 years by numerous persons of varying knowledge and ability. In doing maintenance coding and writing the missing documentation I learnt heaps about geology and mining practices.

I went and did a PhD in software engineering after a couple of decades working in industry and now back working in industry. Thus, I offer my comments in hindsight.

The primary focus of academic research is to extend the knowledge and disseminate that knowledge in the form of papers. Researchers are assessed on the basis of the number of papers they write and grants that they procure. The evaluation of the work is very much a result of the peer review process based on those papers and not the programs that might have been written in support of the core research.

The general observation is that most programs written as part of some research project are a hack written to address a very specific need and fast. A common gripe is that much research is difficult to reproduce because the programs are not freely shared and/or written for non-portable environments.

> A common gripe is that much research is difficult to reproduce because the programs are not freely shared and/or written for non-portable environments.

This indeed is something which I've noticed and what my supervisors have made a great concern. That is, if I create hacks, I should also rigorously document how to reproduce the results. Luckily, much of this comes naturally during the paper writing process which reduces the amount of labor. Moreover, I think many times I end up learning something new or general inefficiencies in my working processes when writing documentation.

I guess I will know better later the year whether academia appreciates the results of my research. Though this far, I feel much of it is like selling in general, i.e., how you frame the results as a part of the bigger picture. And I guess the peer-review process is good in the sense that someone will call you out if you try to build a claim which is completely ludicrous.

Widely generalizing, because it obviously depends on the specific field/university/...:

Academia typically doesn't care about code: sure, you can hack on your open source code, but don't count on funding for that or it being recognized as something as "worthy" as a paper. In extremes: "Why would a Postdoc in a computing related field need to be able to write big code projects, isn't that what students are for?".

Given that, there typically isn't a "programmer" role in academia (although there are exceptions): You're expected to follow the academic path, which involves publishing etc. If your contribution is "I write great code for projects happening here", that's not something that allows you to progress, if it results in a long-term role at all, even if it actually makes others a lot more productive. There's mostly only space for academics who happen to also code, and I've seen a few people crash out due to judging that wrong.

If you want to do research in an academic environment as a job, that's great (although depending on what it is, an industry research lab can also be an option). For people that are more about "making things with code" and don't need to environment, it's probably not the right place. For those that are happy at slowly chipping away at a problem, without pressure, a day job in industry and doing the development on their thing in their free time is also an option that avoids all the problems of an academic career.

E.g. for me personally, I decided to not go for a PhD because I didn't have a burning question I really wanted to solve that a PhD would give me the opportunity to figure out, and without that as a motivation it felt like all the things coming with it wouldn't be worth it (for me, having to do lots of academic writing was the main sticking point. Can't do that well, could probably force me to learn it, but would be unhappy with it). On the other hand, I know lots of people for which that worked out differently, but programming typically isn't the main aspect of what they're doing.

Thanks for the warning! To elaborate, I have come to understand that code is just a part of the paper, seldom the beef. I basically create proofs of concepts, which I've told increase the impact of the paper (thus be fine to work on). I find that in general, it's not about how much or elegant code I write, but whether it allows me and my peers to learn new.

Industry labs are a good point, but I think I have to consider those later when my academic profile is stronger. Beyond Google X, do you know other options?

And I do have the burning question, which is the end-goal mentioned in the OP.

Thanks for the comment, I feel like there's a pressure for me to create a startup (market validated, employees and funding found) but I feel like academia is actually what I want. Your points solidify those thoughts.

My main issue with academia was the relatively poor prospects (the pay isn't great) and the insecurity (there are far far less permanent positions/grants than the number that want them etc.)

To be honest the pay isn't such a big deal as at least in Europe it is reasonable although not exactly generous.

But the insecurity is crazy - like you can spend ages becoming some expert in some area of Physics just to be kicked out in your thirties and while most of the people I saw that happen too didn't turn out badly, you wonder how much more successful they might have been had they just pursued industry from the start.

I think these issues combined with the fact that programmers can get a pretty good deal in industry probably lead many not to pursue careers in academia.

You can take a look on http://matt.might.net/articles/ He wrote quite some blog posts on the academic life of CS-related fields.

In academia most code is useful until the paper gets published, and there only a cursory code review afterwards. It's typical to reinvent wheels (due to the nature of research, it is often not possible to predict future needs/directions so most code is not meant to be long-term). Lots of new people coming in continuously needing training / outside CS most of your colleagues will be beginner programmers. And writing grants / networking / giving presentations becomes distracting after a while.

For me it is the money.

I think I would enjoy it, doing the research, even the grant/publication game. But where I live, I probably wouldn't have money for feeding my family. The money you get kinda assumes you are still a student living in a dorm, or equivalent, unless you are lucky to work on a really well funded project or your academic institution can supply you with a wealth of teaching/other side-jobs.

Compared to being employed as a developer, I just don't see it worth it.

Poorer pay, less interesting stuff to work on and tbh from my own experience, the guys currently there aren't very talented. A fair few had never even been in industry and it showed (e.g. not believing in source control/management).

1) Jog insecurity 2) Lower pay 3) fewer engineers to work with

In that order