Comment by jzila | Hacker News Reader

Comment by jzila | Hacker News Reader

My girlfriend is a PhD student in a pharmacology lab. I'm a software engineer working for an industry leader.

Once, she and the lab tech were having issues with their analysis program for a set of data. It was producing errors randomly for certain inputs, and the data "looked wrong" when it didn't throw an error. I came with her to the lab on a Saturday and looked through the spaghetti code for about 20 minutes. Once I understood what they were trying to do, I noticed that they had forgotten to transpose a matrix at one spot. A simple call to a transposition function fixed everything.

If this had been an issue that wasn't throwing errors, I don't know whether they would have even found the bug. I've been trying to teach my gf a basic understanding of software development from the ground up, and she's getting a lot better. But this does appear to be a systemic problem within the scientific community. As the article notes, more and more complicated programs are needed to perform more detailed analysis than ever before. This problem isn't going to go away, so it's important that scientists realize the shortcoming and take steps to curb it.

i'm in a similar position to you (although i started out as an academic i've worked in the software industry for ages and so end up helping my astronomer partner).

anyway, i disagree slightly with your analysis. in my experience academics know that they suck at the "engineering" part and, to make up for it, are very diligent in making sure that the results "feel right". so i don't think what you described was luck - that's how they work.

in comparison, what drives me crazy, is that if they learnt to use a few basic tools (scm, libraries, an ide, simple test framework) they could save so much time and frustration.

[related anecdote: last year i rewrote some c code written by a grad student that was taking about 24 hours to run. my python translation finished in 15 minutes and gave the same answer each time it was run (something of a novelty, apparently)].

Not sure how your anecdote relates to the conclusion. Forgetting, or even knowing why, to transpose a matrix is not an example of a problem that can be solved by "a basic understanding of software development". Hell, I'm sure there are many decent hackers that don't know what a matrix is, let alone spot such errors within a long sequence of computations.

I suppose the tie-in is simply that we hackers think differently from scientists. It's much easier for us to visualize a complex tree of logic than for people who are not accustomed to it. The second purpose of the anecdote was to illustrate that there is a complete lack of software testing knowledge within the scientific community, or even the recognition of the need for it. All of the "testing" they do is on production data. There is no unit testing anywhere.

I bet that they would have been more likely to have found the problem themselves had the code been cleaner and better documented.

I disagree. Well written and abstracted code makes mathematical formulas easy to scan and proofread.

Your code should look like this:

    angle = acos( ( a . b ) /
                  (|a|*|b|) )

If it looks like this . . .

    angle2 = Math.acos((a.x*b.x + a.y*b.y + 
      a.z*b.z)/Math.sqrt(a.x*a.x + a.y*a.y + 
      a.z*a.z)*Math.sqrt(b.x*b.x + b.y*b.y + 
      b.z*b.z))

. . . you're likely to miss the error*

Bad code compiles. Good code works right. Great code is so obviously right you don't have to wonder.

*Those are the same formula, though the second one is missing some critical parentheses. I use the example because I have done exactly this and been bitten by exactly this, and now am fanatical about keeping my mathematical formulas clean and obvious.

The problem I see with your girlfriend's program is more of a "verification" issue.

In the simulation sub-field I am there is this "research development process" which includes "verification" and "validation" after the model is performed.

Part of the verification is done by "third party code reviews" in which a party unrleated to the program/project reviews the model description (word document) and does a line-by-line analysis of the code to see that the program matches the code.

I did that during my PhD (a Professor at INSEAD paid me to do a code review of a model).

In the case of your girlfriend's lab, they catched the error via "face validation" (the results looked wrong).

Reply on news.ycombinator.com