Feynman's concept of scientific integrity
paulromer.net
paulromer.net
'For a successful technology, reality must take precedence over public relations, for Nature cannot be fooled.' - Richard P. Feynman [1] [1] - http://science.ksc.nasa.gov/shuttle/missions/51-l/docs/roger...
That ship has sailed.
https://en.wikipedia.org/wiki/List_of_things_named_after_Ric...
http://paulromer.net/stigler-conviction-vs-feynman-integrity...
And perhaps it is, but if not, it makes sense to call these specific requirements that Feynman came up with "Feynman Integrity".
Now, because we aren't (intrinsically) proponents of either, we qualify both so as to distinguish them. Either way, the renamed title is far better. "Feynman's concept of scientific integrity". Excellent title.
On a more important note, this approach of being the greatest critic of your ideas is probably a phenomenal way to make fault-tolerant software (not necessarily accounting for every eventuality, but being aware of as many as possible, and appropriately choosing to handle or not to handle them). I'm not nearly as good at it as I'd like to be.
Whether there needs to be an intuitive model behind a theory is another matter entirely. If a theory has predictive power, it's in some sense good, although it may not be doing what you think it's doing. Artificial neural nets have the same problem - they can work, but there's no understandable model of what they're doing.
http://krugman.blogs.nytimes.com/2015/08/02/freshwaters-wron...
Whereas in economics, where the data is fuzzier, and you have questions like, how should the pie be divided up, people are not united together in the pursuit of a universal truth, things are more dialectic. Even the name economics is part of this agenda-pushing, as is the B.S. "Nobel Prize for Economics" and so forth - in the early days, "economics" was called by its proper name "studies of political economy". Nothing was more political than dropping the word political from the field.
We lack the means to experiment for it to make sense anyway. However, we do have the benefit of observation and can confirm predictions through ongoing data collection.
There is enough there to qualify models and inferences made to justify a Feynman style of integrity.
Also in the documentary "Inside Job"[2] is clearly stated that "neoliberal school" (Chicago or New Chicago or whatever) is pushed through universities as the mainstream economic thinking.
[1] http://yanisvaroufakis.eu/2013/10/14/economics-nobel-2013-an...
Short of taking classes at a college (which I doubt I have time for), I don't know how to deal with conflicting things. We see this in talks about whether it's Good or Bad to raise wages, or tax people in various ways, or what to do with the national money supply. All the economic schools seem to have conflicting theories of what should be done, and I don't know how to compare them.
It doesn't seem like they can all be right, can they?
Note that some of the conflicts are not empirical; while economics is a field of science that deals with the relations between observable, measurable events in the real world (though, like many fields, there are can be serious challenges in unambiguously measuring the events of interest), the various schools of economics are not necessarily particular empirical camps with different reads of the evidence, they are in many respects different camps of moral and political philosophy with different views of goals of the economic system. As such, even in matters where the facts are not in dispute, their recommendations will differ.
(This often becomes even more clouded because much of what is publicly presented by public pundits on economic matters is not honest intellectual debate but propaganda by one camp or another to convince adherents of the philosophical positions of the other camps that their different preferences would be best served by policies which are, in fact, preferred for other reasons than those being used to sell them.)
Let's just pick on one area, education.
https://en.wikipedia.org/wiki/Constructivism_%28philosophy_o...
> After a half century of advocacy associated with instruction using minimal guidance, there appears no body of research supporting the technique. In so far as there is any evidence from controlled studies, it almost uniformly supports direct, strong instructional guidance rather constructivist-based minimal guidance during the instruction of novice to intermediate learners. Even for students with considerable prior knowledge, strong guidance while learning is most often found to be equally effective as unguided approaches. Not only is unguided instruction normally less effective; there is also evidence that it may have negative results when student acquire misconceptions or incomplete or disorganized knowledge
So .. the big "theory" in education is that "students learn better by teaching themselves". That's probably not untrue, for a bored upper-middle-class student who's ahead of the class (who do you think does a PhD in education?), but for most students it turns out to be a bad idea.
But it feels good to say "Teach the students how to learn, not what to learn!". And if you can write hundreds of thousands of words on the topic, and give a feel-good summary to peers, what's stopping the idea from gaining traction?
Your observation on constructivism is not in disagreement with that statement.
For an example from medicine, which is also fuzzy, and quoting from http://www.mayoclinic.org/diseases-conditions/menopause/in-d... :
> Hormone replacement therapy — medications containing female hormones to replace the ones the body no longer makes after menopause — used to be a standard treatment for women with hot flashes and other menopause symptoms. Hormone therapy (as it's now called) was also thought to have the long-term benefits of preventing heart disease and possibly dementia.
> Use of hormone therapy changed abruptly when a large clinical trial found that the treatment actually posed more health risks than benefits for one type of hormone therapy, particularly when given to older postmenopausal women. As the concern about health hazards attributed to hormone therapy grew, doctors became less likely to prescribe it.
with http://www.webmd.com/menopause/features/hormone-replacement-... saying "And now, a number of recent studies show that hormone replacement therapy may have protective benefits for women who are early in menopause."
Another example might be ulcers, which a few decades ago were believed to be caused by diet and lifestyle, rather than H. pylori. A third might be the relatively recent work of Ioannidis titled "Why Most Published Research Findings Are False".
You write "For every sociologist, there's an equal and opposite sociologist."
On the topic of education, my understanding is that a large majority of sociologists believe that "socioeconomic status [is] a strong predictor of children’s educational and social outcomes" (quote from http://cepa.stanford.edu/iies2012 ). This view derives originally from the Coleman Report in the 1960s and hasn't appreciably changed over the decades.
If your statement is correct, then there should be a large number of sociologists in education who believe that socioeconomic status is at best a weak predictor. Where are these sociologists?
You're talking about an observation, not a model or even a qualitative explanation.
People have always known that the Sun "moves around" the Earth. That's an observation, and was never in dispute. The model has changed, though.
Doctors have always known that ulcers exist. Their model of the cause changed in the light of new evidence.
Did the Coleman Report do much to change the views of sociologists? I'm guessing they always knew that "rich kids do better at school". Do they think that's due to genetics, teacher behavior, better schools, parents reading to kids? Do they suggest the same interventions, to fix the inequality?
It's certainly a model, based on correlating many observations. The report is at http://www.scribd.com/doc/89990298/Coleman-Report-Equality-o... and the technical explanation at p571 looks very much like the model building I'm familiar with in drug design.
Quoting from http://pages.jh.edu/~jhumag/0400web/18.html :
> Using data from over 600,000 students and teachers across the country, the researchers found that academic achievement was less related to the quality of a student's school, and more related to the social composition of the school, the student's sense of control of his environment and future, the verbal skills of teachers, and the student's family background.
That report (p. iv) deliberately did not suggest changes to social policy. However, it was used to guide changes. Quoting now from http://www.encyclopedia.com/doc/1O88-ColemanReport.html
> First, it showed that variations in school quality (as indexed by the usual measures such as per pupil expenditure, size of school library, and so on) showed little association with levels of educational attainment, when students of comparable social backgrounds were compared across schools. (Differences in students' family backgrounds, by comparison, showed a substantial association with achievement.) Second, a student's educational attainment was not only related to his or her own family background, but also (less strongly) to the backgrounds of the other students in the school. These findings had clear implications for social engineering: opportunities could best be equalized via strategies of desegregation of schools (for example by busing). They challenged a major plank of Lyndon Johnson's vision for the Great Society; namely, that increased spending on education could rectify social deficits.
To answer your question "Did the Coleman Report do much to change the views of sociologists" - yes.
Your earlier statements suggested that you knew something about sociology and it's historical mistakes and developments. Your questions now, which are easily answered and are seemingly a standard part of sociology (Coleman's work is on the Wikipedia page for Sociology), suggest otherwise.
Going back to my question, what is the equivalent in sociology to proposing that a perpetual motion machine can work?
I gave an example - that a human culture can remain unchanged over 500,000 years.
Are there sociologists who believe this to be true? Could I, though force of argument, convince a large number of sociologists that this is true? I believe that is the basis of your opposition.
I do not believe so. There is neither biological nor archeological evidence to suggest this is the case.
If you do not like that one, what is your equivalent?
Another big study was the one that found Direct Instruction (a slightly faddish, but still fairly conservative / traditional model - "tell 'em, show 'em, make 'em do it themselves") was the best fad evaluated, by a significant margin. But for some reason, education seems to favor a model in which teachers try to mimic the heroic Robin Williams in the Dead Poet's Society.
Education reforms are dominated by a few main threads, most of which are rubbish (if you look at the evidence) - the managerial Econ101 (www.joelonsoftware.com/items/2006/08/09.html) type model (reward teachers for good test scores - politicians generally like this one), technology (the obvious driver being the people who want to sell the solutions), and an eclectic collection of "teach students how to think" hogwash (yeah, nice idea, but it's all just mother-love statements) pushed by academia.
I guess I should back off a little from my assertion. Yes, sociologists do ditch bad theories, eventually. But there's a lot of random noise, and the time it takes them to adjust their theories means that the slight signal from good studies tends to be downed out by noise.
Yes, I really think you could convince people that culture is unchanged for 500,000 years. But you'd need a good sales pitch. Here's the skeleton:
* Culture has remained unchanged for 500,000 years.
* If this is true, then it's a moral imperative that people believe this.
Finding a decent excuse for the first part is easy enough. Wave your hands a bit, redefine culture, and find a few examples of things that haven't changed too much.
The second bit is harder, because your proposal is a little boring. A good lie is like a perpetual motion machine - it can keep itself going. I suppose you could argue that exceptionalism - the myth that our culture is fundamentally different, is the cause of a great many travesties.
So, all cultures are fundamentally the same, bar some insignificant surface differences. One of the few big differences (and it's been this way for 500,000 years) between them has been whether they considered themselves special or not (and the cultures which considered themselves special all did horrible things).
The next step is trying to tar-pit people with some kind of paradox - is it OK to consider your culture special, on the basis that it's one of the cultures that doesn't consider itself to be special? But just be a little sneaky with this bit - you don't want people to catch on.
Quoting from the jh.edu link I gave:
> As a work of sociology, the Coleman Report was full of subtleties and caveats, but the mass media and makers of policy focused on one prediction--that black children who attended integrated schools would have higher test scores if a majority of their classmates were white.
> That last point is key because in 1975 Coleman concluded in a new study that busing had failed, largely because it had prompted "white flight." As white families fled to suburban schools, the report concluded, the opportunity for achieving racial balance evaporated.
> Political support for busing quickly waned. Many civil rights leaders, educators, policy-makers, and sociologists who had embraced Coleman's earlier findings now were outraged.They blasted him for abandoning his earlier commitment to desegregation. Some members of the American Sociological Association even moved to have him expelled, albeit unsuccessfully. (Coleman was elected president in 1991.)
> His supporters called him a true scientist who changed his opinion when empirical evidence required him to do so.
You also wrote "..for some reason, education seems to favor a model..."
It seems you have switched from sociology to education policy. Eduction policy in modern US is driven by politics much more than it is driven by science.
You refer to Econ101, so I'll counter with Parkinson's law of triviality, a.k.a. 'dog shedding'. Everyone who makes education policy went to school, and many think that gives them insight into how education work.
1) The social scientists tend to make heavier use of Econometrics, which tends to hold less water the mathematics used by Physicists. [0] [1] [2] This doesn't mean that Econometrics should never be used, just that the tools are tougher.
2) In social sciences, people react to the existence of studies. (Example: Once a paper shows a market inefficiency - frequently it goes away. This could be because it wasn't a real anomoly, or it could be that people trade away the efficiencies)
3) Physics isn't beyond reproach either. To quote Max Planck, "A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it." (I've also heard this as "Physics advances one funeral at a time.")
[0] http://crab.rutgers.edu/~goertzel/mythsofmurder.htm
[1] http://www.nber.org/chapters/c11691.pdf
[2] http://digitalcommons.lmu.edu/cgi/viewcontent.cgi?article=16...
I just read A Brief History of the Post-Autistic Economics Movement (http://www.paecon.net/HistoryPAE.htm). I'm a card-carrying member of this movement and I didn't even know it existed.
Romer thinks the reputation of his whole field is at stake if they don’t agree to some common norms about the purpose being to seek the truth, and he’s therefore calling out economists who he thinks don’t abide by the norms he prefers.
Of course it’s “politics”, in the same way it’s “politics” when an experimental physicist calls out string theory for being unfalsifiable.
Not really. It's as though he's calling out someone for presenting results that show the Earth rotates around the sun, but then in the text of the paper says the results show the opposite.
Anyway, his primary point isn’t about “mathiness” per se, but rather about the way (he claims) a particular group of economists refuse to admit that their theory turned out to be wrong when put to real-world tests. The “mathiness” part is just one of the ways he thinks they try to deflect criticism through confusion/obscurity.
(Note, in my comments here I’m not judging the validity of any of these arguments one way or the other.)
Side question: this is driving me crazy and Googleing isn't helping. Is it "beyond repute" or "beyond reproach"? Are both OK in this context?
Others have said this too. It may be that more substantive stories demand reflection before comments arise. Conversely, shallower posts often trigger a flood of comments in a few minutes. That's usually bad because such reactions are more reflexive than reflective, typically served from cache with little attention to whatever is specific about this story.
The difference in reaction times is noticeable, like a timing attack on the brain. Hmm... we could measure this.
One can sum up everything we're striving for on HN as s/reflexive/reflective/.
"This evidence strengthens my belief that the fundamental divide here is between the norms of political discourse and the norms of scientific discourse. Lawyers and politicians both engage in a version of the adversarial method"
In law there is a case to be made for the adversarial method (Though the disparities in what each side can afford in legal counsel undermine it almost entirely), but I question whether it is good for politics. We are so polarized and reflexive in all debates. Libertarians refuse to concede any flaws in their model of how things work, and the same is true for those who want the government and laws to take over.
The question is, has social progress mostly come about through political war, where each side employs any means necessary to win, including dishonest ones, or through honest discourse, where we arrive at truth not through attrition but through reason? I want it to be the latter but most of the world acts like it's the former. And quite often the appalling injustices and suffering that continue in the world cause me wonder whether I should give up idealism and grab a musket.