Comment by denisrosset | Hacker News Reader

Comment by denisrosset | Hacker News Reader

Parent Full thread

> The culture and dynamics around scientific and academic programming is different from both FOSS and commercial software. This needs to be taken into account.

I'm currently teaching modern scientific Python programming, if that's a thing.

You raise an excellent point, the software development life cycle is quite different in academia. Do you have resources in mind on that topic?

I was thinking of dividing academic code in 3 "Ps":

- Playground: throw-away scripts for data analysis, fast moving code where any rigidity slows the process. Internal use by a single researcher usually.

- Prototype: code that serves as a foundation to the above, or is reused frequently, but is otherwise not shared outside a research group.

- Product: code, often open source, that is shared with the community.

Most tutorial about "good coding practices" do not make distinctions between these three stages.

For Python, there are practices that work at any level: auto-formatting code and sorting imports, gradual typing. But things like tests, API documentation, dependency management only apply to steps 2 and/or 3.

Stage 3 does not always happen. There are a few remarkable counterexamples, but most research software does not usually finish taking the form a product.

In my experience, what happens is that some prototype works quite well and gets used in the next project. But the next project is building a new prototype, not improving the previous one. After a few projects, the person who wrote that first prototype already left the department, but there are people who depend on it to get results. If enough time passes, you will have some piece of legacy code that very few people can understand.

The problem with academia is that, while the prototype is an excellent topic for an article, the real product, a maintained (and maintainable) software project, is not. And getting funding in academia to do something of no academic interest (ie. that cannot be turned into a paper) is very difficult. If the software is of no interest to some industrial partner willing to spend money, that code will just keep moving between the USB drives of PhD students, probably branching into slightly different versions, until someone is brave enough to start working in a new prototype and the cycle starts again.

Maybe add a fourth category.

- Petrified: code that's set in stone and everyone's to scared to touch in because "it works, but God knows how".

With the increasing expectation that academic research results should be reproducible, there's really no such thing as throwaway "playground" or "prototype" code. Everything is potentially a "product".

Not only are software development life cycles different, but also the attitude to development. Which is not a surprise when in most cases, the output is not the software (or system), but research or learning; the code is merely a side effect.

Reply on news.ycombinator.com