Hacker News Reader

Comment by akashshah | Hacker News Reader

This sounds like best plans for computing in general, not just scientific computing

Yeah for scientific computing specifically i was more expecting to see concerns like ensuring intermediate working state is persisted to a durable medium and facilitating restarting from a working state "dump" instead of from the top each time.

Around the end of summer there was a post on just this topic. I can't find it now but it had really solid practices in this area. If anyone remembers it / finds it I'd love to be able to read it again.

You may be talking about a post by S. M. Ali Eslami called Patterns for Research in Machine Learning[1]. There was some discussion here as well[2].

The patterns he pointed out were:

  1. Use version control.
  2. Separate code from data.
  3. Separate input data, working data and output data.
  4. Modify input data with care.
  5. Save everything to disk frequently.
  6. Separate options from parameters.
  7. Do not use global variables.
  8. Record the options used to generate each run of the algorithm.
  9. Make it easy to sweep options.
  10. Make it easy to execute only portions of the code.
  11. Use checkpointing.
  12. Write demos and tests.

EDIT: Added list of patterns, formatting.

[1] http://arkitus.com/PRML/

[2] http://news.ycombinator.com/item?id=4384317

Reply on news.ycombinator.com