Hacker News Reader

Comment by maurycy | Hacker News Reader

Finally. Finally a discussion about this.

Enough with the false melodrama, please. Aside from the fact that your comment is content-free and inane, scientists have been discussing this subject since computer simulation first became a part of science. A lot of scientists do share their code (I'm one of them, and I believe in sharing code). But there are good arguments on the other side. Among them:

1) Papers describe methods in enough detail to reproduce them. If they don't, there's a serious problem.

2) Independent lines of verification. If simulation code becomes a reference, it's inevitable that the same bugs/bad assumptions will contaminate an entire field. Independent re-implementation of the same algorithms is a strong hedge against this phenomenon (even if it means that there are more bugs overall).

3) Money. A lot of scientists fund their research in part through licensing of implementations of their algorithms. I don't like it, but until someone gets around to repealing Bayh-Dole (a real scientific travesty, IMO), this is going to continue to be a problem.

In short, what you really meant to say was that finally someone wrote a newspaper article about this subject. It's not a new discussion.

I'll add another argument for not releasing scientific code, specially of a matter so polarizing as climate change: programmers aren't scientists. We aren't familiar with neither the field's literature nor the science being used. For example, the famous comment from the CRU code said:

; Plots 24 yearly maps of calibrated (PCR-infilled or not) MXD reconstructions ; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually ; plot past 1960 because these will be artificially adjusted to look closer to ; the real temperatures.

Do you expect the average programmer to take a 2 year course in climate science to truly understand what MXD means instead of instantly running around screaming "wolf!"? If you did so, be disappointed (a careful reading of Google results for "CRU code" might be enlightening).

That seems like a bogus reason. Some people who had already decided that climate change was fraud decided to read comments in the code and interpret them to fit their worldview.

Other people read the CRU code and made intelligent comments about the code.

Hear, hear! I did a whole series of posts analyzing a skeptic's claims:

http://matthewmarkus.posterous.com/raw-darwin-airport-temper... http://matthewmarkus.posterous.com/darwin-airport-temperatur... http://matthewmarkus.posterous.com/darwin-airport-temperatur...

I found at least one error/discrepancy in each side's output. Of course, neither side released any code. The disclosure of methods is a prerequisite for repeatable experiments, the cornerstone of science.

Closed academic publishing is intellectually bankrupt, and is probably one of the greatest problems effecting research today. People don't share code, and put a paywall between themselves and the public. There are open journals, but they are rarely as prestigious, and so are not as valuable to those seeking tenure. These academics put tenure before fruitful scientific discussion.

A respected journal needs to be able to pay for someone to review a paper before publication. I suspect that setting up a foundation to do this for free would be a great charity, but without some sort of backing you can't create high quality.

In nearly all fields, referees of scientific papers are not paid. Referees of scientific books may be paid a small honorarium, but, compared with consulting, it's a pittance.

Background experience: I refereed somewhat over 100 papers and perhaps a dozen or so books during my career as a physicist. My work now overlaps with the scientific publishing industry more broadly.

It is my understanding that referees are sometimes paid, but editors rarely work for free. (Using http://en.wikipedia.org/wiki/Peer_review version of editors.) It's a lot of work to find out who is best to referee a given work and to track down conflicts of interest etc.

With editors the issue of payment depends on the journal. Some journals employ a staff of professional editors. Others recruit tenured scientists who do the editorial work for an honorarium that seems tiny, considering the amount of work involved. I don't know which model is more widely used - I can think offhand of many journals of both types, but don't recall ever having seen statistics.

I agree in general, but wanted to add one small caveat that I think is interesting: PLoS Biology is (a) open access, with a Creative Commons license; and (b) has rapidly established a reputation as one of the top journals in biology.

So would you rather people publish in "low-impact" journals and then leave science completely because they can't get a permanent job?

"Intellectually bankrupt" is a pretty strong term to use for people who work for a small fraction of the amount of money normally talked about on this site.

I'm not saying there aren't issues, but blaming the individuals who are trying to make a living by doing science isn't going to help. The success rate of getting permanent jobs in science might be higher than that of startups, but the "payoff" is a small fraction.

I have not left science completely: I've made my own job. It is possible but it is only made harder because of the closed system.

There are many of us who've left academia and still do science. We're generally maligned, and removed from the ability to even participate in a discussion due to a variety of academic access restrictions, and why?

What's more, day by day people are showing how to achieve scientific credibility and influence through their blogs and paper hosting services like ArXiv or, as Michael Nielsen points out, open journals like PLoS Biology. The majority of scientists still bow to tenure pressure, and frankly I don't understand why. There are other opportunities if you want to gain status, and one doesn't even have to gain traditional academic status if one wants to do real science. There are other options.

Which academic access restrictions are you talking about? I know people who have started independent "institutes" but the only reason you need to do so is to receive federal funding. It's true that if you brand yourself as an "independent researcher", people might be inclined to think you are a crackpot, but publishing real papers should take care of that.

I'm not sure blogs are a relevant source for scientific studies though. Not necessarily because I think peer review is the greatest system, but having your paper published in an actual journal (open journals are fine) at least means you managed to convince a few other people that it's worth looking at the paper.

Those are bad reasons on all counts:

1) Code is notoriosly difficult to describe in a natural language, so it's unlikely papers contain enough detail to replicate a complex program. 2) People can still reimplement the model even when it's available. In fact it's going to be more likely that non-scientists join the convesation (which will improve the abysmall public perception of science, especially climate science). 3) Making money is great for an inventor and/or startup founder. You can still sell implementations, but they can't be trusted until they have been run on a large number of independent computers. Until then, it's not science.

Reply on news.ycombinator.com