The Rank Amateur
jgc.org
jgc.org
Science is actually admirably egalitarian. That's not one of the things I think is wrong with it. The heavily politicized branches might not be egalitarian; I have no experience with them. But in general, you don't need to be a member of any professional organizations or have any other credentials to submit your work to journals and conferences (which is how we transact our business.) Many journals employ double-blind reviewing, in which case even unintentional discrimination based on credentials is unlikely or impossible.
Nevertheless, scientists might often give the impression of not wanting to let amateurs into the 'club'. Why is this? It is simply an issue of bandwidth. Think about this: for each amateur, like the author, who had something to contribute, how many do you think thought they had something to contribute but were mistaken? If you guessed a hundred thousand you'd be in the right ballpark. I've seen it in many different areas.
For example, the number of people trying to submit "proofs" of P != NP, (or worse, P = NP) is just ridiculous. Some are well-intentioned although ignorant, and others are just cranks. Some fields of research attract more amateur claims than others, but whichever field you look at, the amateurs greatly outnumber scientists, and the vast majority of them are mistaken.
So what do we do? We use a simple filter. If you speak our language, we'll listen to you. This is what appears to the lay public as anti-amateur. But it's not, really. All you have to do is to learn some simple definitions and terminology set forth in a straightforward way in previous papers, to prove that you've done your homework. Then write up what you have to say and we'll be happy to give it a read. Sure, like every heuristic, it's not perfect; sometimes there are false negatives. But there really isn't an alternative. Without a filter, all we'd ever be doing is debunking crackpot theories.
I acknowledge that much of this probably doesn't apply to climate change "science," which is a special case. But there's been a lot of criticism of science in general, and I wanted to set the record straight. It's important to distinguish between what's really broken and what appears broken, otherwise we risk throwing the baby out with the bathwater.
We have the internet now. There is no reason to not put up a site with data & code when you publish your paper and just point all requests to go to the site. Scientists need to start using these tools like the rest of us and stop hiding behind such false excuses.
I agree freedom of information requests should always be respected (if nothing else it's the law). That doesn't mean it's useful on it's own.
Unintelligible source code isn't much more useful than none. See CRU's own source code for example: http://www.jgc.org/blog/2009/11/about-that-cru-hack.html
It's nice that some spend the time to clean up code and format data. But scientists progress on their career track by total number of papers; so they're being somewhat asked choose between the greater good and their own.
Most research is publicly funded. If open access is important to you then write to your government representative and ask them to require it.
Every once in a while we'd get a "walk in", someone with something he came up with, thought was brilliant and that "the people at MIT" would be interested in. Our lead professor was quite gentle with these guys, but the outcome was clear ... and part of it was certainly that they didn't "speak our language", which when you come down to it is a pretty low bar.
Low in that reading some of the literature and sitting quietly at a few gatherings would easily bring you up to speed.
I actually spent 3 years doing research on the other side of the paywall. It is one of the things I'm "vocally critical" of. Here's me calling it evil just a couple of days ago: http://bit-player.org/2010/yet-another-spam-update#comment-2...
At any rate, there's not a single scientist who believes that publisher copyright for papers is a good thing. Unfortunately we can't change the system overnight. At least in CS, as I pointed out in that comment, it has already become a non-issue.
Furthermore, people are ludicrously impressed by 'PhD's' from 'top schools' with 'impressive publication records.' Even a former academic with no institutional affiliation falls off the radar screen, and so those who want to be taken seriously in science are goaded into a costly, inefficient, and decades long rat race, just so that people will take them seriously.
He finally gets an academic to meet with him to hear his ideas but won't accept it when the academic points out the flaws in his reasoning. Throughout the story the electrician insists that the math isn't important, that he's right even though he can't express the math behind his ideas. He thinks the academics are being elitist (or, as you write "anti-amateur") just because they insist on using math.
It's a bit like the situation with date of birth and CVs: you can figure out how old someone is, if you want to know that, by looking at their degree year.
Additionally, referees can sometimes give themselves away.
The authors also fail to cite [Irrelevant Paper, Irrelevant Paper 2, Peripherally related paper 3]. These works should be cited and compared to the work being reviewed.
"Irrelevant Paper", John Smith. Journal of Trivialities, 1995.
"Irrelevant Paper 2", by John Smith and Steve Jones. Journal of followups to irrelevant papers, 1998.
"Peripherally related paper 3", by John Smith and Phil Johnson. Journal of Blah Blah, 2002.
Well, he's not saying that a scientist is always right and an amateur is always wrong. He's just saying that a scientist is more likely to find errors in research material, which I think sounds reasonable since they're supposedly trained to do it. Then there's e.g. the good ol' "seeing what you know is there instead of what's actually there" effect which works against those who've spent a lot of time with something.
Good find on the bad data, btw! Real science is in the numbers, not in the propaganda surrounding them.
Climate science is not in its infancy. It's broad, deep and complex.
I desperately hope you're not implying that we should give credence to the din of crackpot circus clowns who think dramatic climate change isn't really happening and that Al Gore is personally placing space heaters on the glaciers and polar icecaps.
You are being downmodded because you are nonsensically comparing them to deranged conspiracy theorists. You do so without evidence, most likely for political purposes.
I have no political purpose -- and I'm not sure how politics play in watching shipping lanes open up on the north pole.
I'm glad if they find errors. But correct me if I'm wrong, it hasn't change anything. And this is what concerns me: that it raises a hand for debate. The debate is over. We need to act and we need to have started yesterday. I feel like I'm watching a woman get raped, but instead of people stopping it, they want to talk about if it's really "rape" if the man wears a condom.
We are at a tipping point and we can, and should, cut carbon emissions. Moving to alternate energy sources is an important step that we will make eventually anyway, but to do it sooner means that the poorest of the world will not get quite as screwed in the end.
But it seems that many people are more interested in having their own pet ecopocalypse to feel self-righteous about than actually getting the science correct.
You're not poor. You're not at risk. It's the water supplies of millions of poor that are vanishing, not yours. It's the possible forced migration of millions that's pending, not yours.
These are not "useless environmental projects". They are changes we will make regardless. But doing them now will save lives.
Some amateurs have identified errors in the published literature. Because you are concerned that their corrections might affect the debate, you compare them to deranged conspiracy theorists in an effort to discredit them.
You do this because you feel that, in light of new evidence, we may not implement the specific policies you wish to be enacted. Then you claim you have no political purpose?
Because you are concerned that their corrections might affect the debate, you compare them to deranged conspiracy theorists in an effort to discredit them.
No, I simply don't want action to stop because an amatuer found a minor error and start having to go back to debating it all from the beginning again.
No, I think we should give credence to the work of amateurs like Steve McIntyre whose high quality criticisms of specific pieces of work have been accepted into the scientific literature and who has been an important force in helping us see more clearly what is happening with the global climate.
But since you seem to have your mind made up, I guess we should just call each other names.
I'm not sure having a PhD qualifies you as a professional anyway - certainly not in the PE, lawyer, doctor or accountant sense.
What is that if it isn't an amateur?
A PhD is a researcher merit badge. Obtaining it means you've been trained by people in your field how to do research. I have real difficulty calling anyone working on their PhD, or anyone who has a PhD, an "amateur."
I was certainly part of the academic community but I wasn't a paid "professional" researcher.
In that context there is a very sharp boundary between "professional" and not - for the rest of us the boundary is rather fuzzy. I certainly don't regard myself as a professional simply because I am paid to do something or that I make my living out of it or even, unlikely as it may seem, occasionally show some competence ;-)
Journalists attempt to generate controversy to increase page views. Myles Allen was correct that corrections to climate models have come from scientists amateur or otherwise not journalists writing copy.
John Christy took a lot of heat over the satellite data, but nothing remotely like what is being turned on Phil Jones. It would have been romantic if John's error had been uncovered by journalists combing through stolen emails, or members of the public issuing freedom of information requests. But it wasn't. It was found by the US government funding a painstaking independent analysis of the satellite record, with John's co-operation, just as Phil has said he would be happy to co-operate with an impartial and scientific re-analysis of the surface temperature record, if anyone wants to fund such a thing.
The implication is that none of these 'members of the public' are any good, the only thing worth considering is funded research.
I also wonder about him invoking his professional capacity. It implicitly makes him a representative of scientists; in acting so he really should have been more nuanced.
In many fields, today, to be an amateur you need to be independently wealthy as well. Most science requires a lot of expensive equipment. The "amateurs" he spoke of were using techniques that required little money (data analysis, pen and paper math)
From what I remember from a biography he always did excellent in physics and math, and in that famous entrance exam (that I think he took a year earlier than usual) he only failed in French, which he didn't care much for.
While you posted a fact (the failed admission test), that fact is a cherry-picked outlier that gives the opposite impression of Einsteins actual academic achievements (especially more so if you omit the rest of the circumstances).
It's like saying the fact that I won my Physics School chess championship in 1996 and leaving it there (true) without mentioning that I'm actually a mediocre player that won because many of my opponents didn't show up.
My rant was more general; I'd swear I've seen in a couple of movies the "Einstein flunked a physics test".
They put the data out there and someone found an error. Someone else might spot an anomaly that leads to some important discovery.
Whether that someone is a professional scientist, a professional programmer, a statistics student or a crank is mostly irrelevant. Putting stuff out for re-use can (if planned for) be cheap and easy. There's no reason in this day and age, not to do it.
That's what's important here and pitting professional against amateur doesn't really move you forward on that issue.
We went to a client to work on Problem X. He quickly determined that solving Problem X would achieve nothing. Problem Y was the real problem, but was way outside our areas of expertise.
So what did he do? He slept 4 hours a night for the next 2 weeks studying everything he could find about Problem Y. He reviewed reports, industry literature, called experts, and talked to as many people in the company who knew anything about that subject area. Within 2 weeks, he presented a brilliant solution that no one had ever considered but was instantly understandable by their experts. (That solution included work done by us and we had a great client relationship for years.)
Later, I asked him why he tried to accomplish something so difficult with such a seemingly tiny possibility of success. I'll never forget his reply...
"I didn't know that I couldn't do it, so I did it."
Note the crucial elements.
He talked to a lot of experts in field Y. This is not a "lone genius"!
He had the freedom and incentives to work on this almost uninterrupted. He would have spent at least 100 hours, which is worth at least as much as two undergraduate courses. Also, he was an expert in related field X, which must have helped.
Furthermore, he because he knew the practical problem, he could cut down the search space. He didn't have to investigate everything about field Y, just stuff applicable to him.
If you consider all that, the solution doesn't look so miraculous. What's more interesting to me is that I think even ordinary people could be achieving that kind of result all the time if they had the right background, incentives, and support.
(HN submission: http://news.ycombinator.com/item?id=1064931)
Good hackers work in all sorts of fields they are not experts in. I can't tell you the number of times I've seen "breakthroughs" with technical problems simply because the smart group with all the jargon had to explain themselves and their situation to the outside hackers just trying to make a computer program help them out. It's extremely easy for groupthink to take over small groups of specialists, and it happens most of the time without any of them realizing it. By the way, just so we don't feel so cocky, this also happens all the time in groups of programmers in a project who take hidden assumptions and make them into unwritten law without anybody questioning them either.
The amateur, outsider, or programmer has a vital role in technical groups and efforts of all stripes.