Cargo Cult Science (1974) | Hacker News Reader

Cargo Cult Science (1974)

fermatslibrary.com

113 pointsBerislavLopac57 comments

Here's a better copy of this classic Richard Feynman address, rendered in text instead of graphics (meaning it can be copied in whole or in part), without any advertising: https://calteches.library.caltech.edu/51/2/CargoCult.htm

This is one of the more influential ideas I've ever come across. Not that the actual behaviour of people living on isolated islands briefly visited in wartime matters so much, but it's a good story. If your take away the concept of

"In trying to emulate success by copying without understanding, you may make the error of copying the visible things, rather than the relevant ones."

then it's a pattern than can be seen in all walks of life, rather than just science (where the difference may be particularly stark). I find that many people will now recognise an accusation of "cargoculting" (although I have heard this interpreted as meaning the worship of a British prince).

In many cases people are being rewarded based on superficial appearances, rather than concrete results. No wonder that's what they optimize for.

I think that there is a pernicious impact on many folks because they do "all the right things" and still struggle mightily with their work and responsibilities because doing 80 hard hours of the wrong work doesn't deliver.

It's also what a lot of AI are optimising for, which is where I think the risk is at the moment: the state of the art in AI has gone from "these things don't have any common sense" to "when common sense and reality diverge, they often go for common sense".

Great for chat, not so great for avoiding surprise botulism in home-made garlic oil: https://news.ycombinator.com/item?id=40724283

The supremacy of form over function, appearance over substance has been a dominant feature of our society for a long time. From the mid 90s:

  """ Just visit Los Angeles. Just walk down the walk of stars at one
  o’clock in the morning, and ask yourself “Are these people really
  here, or is this central casting?” And it isn’t a funny
  question. And as our society develops in this telematic way, it’s
  not going to be funny to ask it around the Christmas dinner table
  about Uncle Henry, "Is that Uncle Henry, or is it just someone
  playing Uncle Henry?" The further postmodern insight is: what
  difference does it make! See, that’s the thing Reagan's handlers
  understood. It doesn’t make any difference! They’re buying it! """
  -- Rick Roderick

Some might say this observation/trend/etc. is much older than the 1990s, as perhaps exemplified by Oscar Wilde's The Importance Of Being Earnest in 1895 (where literally all that matters is some one's name being "Earnest"). (Apologies if Shakespeare or the Greek Comedies or other got there first - feel free to cite!)

Absolutely. I think it's human nature for some to get stuck on superficiality and appearance. Thanks for the reminder of Wilde.

Been that way a long time. Economics ha something called the axiom of revealed preferences. Behavior only shows you what the person decided among their available options.

Quite right, and I'd say this could be considered an instance of the phenomenon known as (among other names), Goodhart's Law. If we consider superficial appearance of success to be a good proxy for success and hence a good metric to optimize for then we necessarily end up moving away from our original goal.

I love this article as it speaks to one of the most overlooked aspects of modern life: There are a lot of fundamental things that we take for granted and practice every day - a lot of things with lots of experts and powerful institutions around them - that simply do not work.

And once you have such a thing, and you've build your reputation, your power base, your institution around it, it is absolutely paramount to carry on with it, deepen it, and extend it - regardless of whether it works or not.

But continuing the thing that doesn't work doesn't require some malicious intent or ulterior motive. Quite often, it's just a question of "well, what else would we do?". And the answer to that is often "nothing" - or "we don't know" - so it's better to do something and feel like we have agency and are doing something about it than just carrying on with our lives.

I'm not sure that "education" as the article states is one of them. But the war on drugs certainly is.

For me, the word "education" does not appear in the article hosted at the URL https://fermatslibrary.com/s/cargo-cult-science

Maybe fermatslibrary.com is a badly designed website that returns SEO spam like content depending on such things as IP address, browser or some other factors.

It's interesting that a URL is no ground truth, and that my comments below are discussing an entirely different text.

Education is mentioned on the first page in the 6th paragraph.

Upon reading your other comment, it's possible that whatever you are viewing the page on is not rendering the pdf containing the actual text and you are only seeing the annotations.

Education is definitely one of them. Kids learn at public schools by accident. Utah has some of the lowest spending on education in the nation and yet it has some of the highest standardized testing scores. Similar to the war on drugs sometimes public education correlates with actual learning, but sometimes arresting people who have drugs lowers drug usage. In both cases we don't actually understand what we're doing.

> a lot of things with lots of experts and powerful institutions around them - that simply do not work.

Part of what's going on is that "does not work" is usually a vague moving target.

"Does not work" can be said about any gap between the goal and where we are currently. If you have a system that, on average, closes that gap over time, then the system "does not work" for the entire time it's running. After all there's a still a gap.

Chesterton’s Fence is a kind of inverse impulse to what you're describing. It may be that we think our current systems are nonsensical, but when we change them we realize we've created new problems that the old systems were solving.

And then there's often disagreement about what the goal should be. He mentions decreasing the amount of crime by how we handle criminals. Not everyone agrees that's the goal of the criminal system. Some people very strongly believe that criminals need to be punished or harmed, and those people vote. But punishment and crime reduction are different goals and they may compete. A solution that reduces crime may not harm criminals enough for the blood thirsty voters, and vice versa.

So my reading on all this is that, yes we have gaps in our understandings of a lot of things. But many of those will not be fixed until there's more agreement about what the goals are.

I agree with that - but in many cases, I think I'd at least define "works" as "is a benefit towards the set aim, rather than a detriment". Although in many cases, it'd be fairer to define it as "is a net benefit towards the set aim when taking all costs and externalities into account".

In this way, I would argue e.g. that

- the war on drugs is a detriment, having significantly increased the price and thus the profitability of drugs and fostered a cartel ecosystem that is now a large percentage of the economy (and often the government) of many countries.

- the war on terror or the prison system might be a net benefit towards the aim of reducing terror or crime respectively, but is a net detriment when taking its costs (monetary, social, freedom etc.) into account.

Aren't those examples pretty well understood to have true motivations that differ from their "marketed" motivations? Those detriments are actually part of the hidden set goal that led to those programs existing in the first place.

* The war on drugs was motivated by racism and marketed on morality and harm.

* The war on terror was motivated by power & military-industrial-profits and marketed on fear.

* The prison system was motivated by punishment & revenge and marketed on lower crime.

It's the social equivalent of WONTFIX: Working as intended

Crowstrike.....

But anyway, there's a famous phrase: "The goal of a system is what it does"

Once you have more than 1 person, any idea that involves "goal", "aim", or a related concept is meaningless. Those things simply do not exist.

That's simply not true, goal of most businesses is to make money. You can clearly deduce this goal much more the bigger the company gets. What you say only makes sense if you are confusing the stated publicly goal, the appearance that company maintains and its true goal which is pursued by the executives (the capital).

How many people in a company have the one goal "make money for the company"? How many have that goal at all?

Companies don't have a goal. They are just a bunch of people.

Yes but the overarching structure they participate in is designed to pursue the goal of making money. If not every single person has the same idea then we cant deduce anything from the group as a whole? When you look at the heap of sand you also start saying things like this is not a heap of sand its just a bunch of sand grains?

Do you attribute intent to the sand? Where are you going with that metaphor?

My point is that there is no clear line between a heap of sand and a bunch of sand grains. But at some point the grains become a heap of sand regardless of this. Its easier for us to think about this multitude as a singular object. In the same way a collection of people as a whole can have goals, we can speculatively discern that even though a company has hundreds of people, all with their own goals and ideas. We can discern the general structure of all of them as a singular unit because it might be a useful insight. If a company as a whole given a decision always goes for the decision which will bring the most capital then we can say that the true goal of this company is to pursue money even though all the people might not be aware of this its still a useful and valid idea that is capable of predicting reality and so on.

If the goal of a system is what it does, then every system's loss function is defined to be the behavior of the system. So every system is optimal.

The conclusion that every system is optimal should give you pause that maybe that phrase isn't as clever as it sounds.

While, as Feynman describes, "Science" absolutely aims to be better, scientists are also human animals and language (or maybe more aptly "imitation") instincts run deep throughout the animal kingdom.

There are pros & cons. On the one hand mistakes like The Cargo Cult propagate. On the other hand, so do skills & shortcuts. There are fine lines between imitating & pretending, practice & pretension. Nature videos show little predator cubs often doing "practice attacks" with siblings... I've no idea what the rate of failing to hold back on a bite is, but presumably the "play" serves most of them well later on in their lives in real hunts.

In short, and very related to Feynman's essay, it's also very easy to fool oneself into thinking one has solved The Demarcation Problem [1] without actually doing so in a way portable/persuadable to other minds. (But Feynman was surely a true genius and would probably have acknowledged the incompleteness of his account in what was after all only a brief graduation speech!)

[1] https://en.wikipedia.org/wiki/Demarcation_problem

Cargo Cult Science (1974) [pdf] - https://news.ycombinator.com/item?id=37107817 - Aug 2023 (136 comments)

Cargo cult science (1974) - https://news.ycombinator.com/item?id=35041549 - March 2023 (156 comments)

Cargo Cult Science (1974) - https://news.ycombinator.com/item?id=26563567 - March 2021 (69 comments)

Cargo Cult Science (1974) - https://news.ycombinator.com/item?id=21120637 - Sept 2019 (33 comments)

Cargo Cult Science (1974) - https://news.ycombinator.com/item?id=13290107 - Dec 2016 (37 comments)

Cargo Cult Science (1974) - https://news.ycombinator.com/item?id=11669004 - May 2016 (38 comments)

Richard Feynman: Cargo Cult Science (1974) - https://news.ycombinator.com/item?id=6543791 - Oct 2013 (62 comments)

Feynman's Cargo Cult Science - https://news.ycombinator.com/item?id=1629571 - Aug 2010 (25 comments)

Cargo Cult Science by Richard Feynman (.PDF) - https://news.ycombinator.com/item?id=993150 - Dec 2009 (4 comments)

Cargo Cult Science, by Richard Feynman (1974) - https://news.ycombinator.com/item?id=723140 - July 2009 (17 comments)

Also related are submissions of https://gwern.net/maze which is about finding the sources for the psychology anecdote towards the end.

Thanks! I found:

Feynman’s Maze-Running Story - https://news.ycombinator.com/item?id=35798239 - May 2023 (46 comments)

> You must do the best you can—if you know anything at all wrong, or possibly wrong—to explain it.

I can imagine a frustrated researcher reading this and muttering, "and what? Risk publication?". Scientific integrity, of which Feynman talks is much more difficult to maintain if the said integrity leads to a substantial decrease in the quality of life. As is known, research is a hit or miss and there will never always be groundbreaking results. But if you are only rewarded for such groundbreaking results, you are then incentivized to package the most interesting aspect of your study while burying away the rest of it.

So long as honesty, not success, is not properly incentivized, science and scientific methods will never prosper to the heights that Feynman talks about.

I wonder how much of a role did the academic conditions play back in those decades. What does a Feynman do in 2020? Probably not compete for tenure.

The science is settled!

The vast majority of us live in a secondhand reality. A kind of culturally secreted dream. It ain't real but it's useful.

> some kind of mysticism, expanded consciousness, new types of awareness, ESP, and so forth. And I've concluded that it's not a scientific world.

Something being not scientific doesn't imply it doesn't exist. It may instead just be too hard to observe scientifically, sound too crazy and appear uninteresting to scientists. Nowadays there already is enough credible research in these areas to be confident exotic phenomena actually take place in peoples brains when they report mystical experiences - thanks to techniques like fMRI, EEG, transcranial stimulation etc. People may tend to make epic claims and construct wonky ideologies based on what they feel, but they don't always make the entire story up. UFOs also are known to exist AFAIK - there just is no reason to believe that's alien spaceships, that's just flying objects we failed to identify and explain. We just need more real scientific exploration to decrease need in fairly-tale explanations of the phenomena people observe - hasn't this almost always been the case with everything since the stone age? The very fact people try to discuss "supernatural" phenomena with a scientist suggests presence of the demand for more science in this.

In the context of the talk/article, I think its clear that Feynman is critical here of the "fairly[sic]-tale explanations", and not the idea itself.

I've always understood this talk as amounting to "we claim that our society respects science, but our society seems unwilling to apply the rigor that science demands to sort out fact from fiction. See, for instance, all of these erroneous explanations for observed phenomenon, explanations that do not hold up to rigorous examination".

Modern psychiatry is a well-intentioned cargo cult. I borrowed from Feynman for my essay at Mad In America: Cargo Cult Psychiatry. https://www.madinamerica.com/2022/06/cargo-cult-psychiatry/

Feynman would have had a field day with the idiots proclaiming "The Science is Settled!"

    > The first principle is that you must not fool yourself—and you are the easiest person to fool.

This phrase is repeated many, many times in the legendary fan-fiction Harry Potter and the Methods Of Rationality (HPMOR)[1], which I'm sure many here are familiar with already.

It is perplexing to hear people making room for their favourite pseudoscience using the excuse that "there are things science cannot explain". See, the annoying thing about science is that if something can as much as be observed, then it can be observed repeatedly, and then reasoned about - effectively allowing us to do science on it. That's even the case for magic, if it were real, as illustrated in HPMOR.

Even the most absurd and chaotic phenomenon, bereft of any discernible pattern or rhyme or reason, can at least be observed to behave chaotically and then be described as such. Et voilà : science !

[1] https://hpmor.com/

Yeah , cargo cult science doesn't work but Cargo Cult Entrepreneurship (CCE™) can make you rich.

Subscribe to my newsletter

"The first principle is that you must not fool yourself - and you are the easiest person to fool."

And this is what a lot of developers do: copying superficial patterns without understanding. It used to be junior (and not so junior) programmers using unnecessarily complex design patterns without understanding what problems they were about to solve. Then it was building systems out of dozens of services to be able to scale, because they didn't have any idea of what kind of scale they actually needed more than one cpu for.

That's how cargo cults start. They way they are propagated, though (in tech), is when devs become old enough to become "senior" without yet having grasped how and when to use patterns, and then proceed to teach them to juniors as mystical, holy imperatives that must be repeated without questioning.

Plenty of businesses have a life cycle like this:

  1. A group of devs strike on some effective patterns through some combination of luck and deep insight.
  2. Juniors get hired and are taught the patterns, but not the WHY of the patterns.
  3. The founders leave, and the first hires become the leads.
  4. The business has now lost the ability to create anything innovative, as everyone are now slaves to the original patterns.
  5. Gradually the business loses the ability to recreate even the original patterns, as some details are lost every time they are handed over.

We should add “- Richard Feynman” to the title. I was pleasantly surprised to notice that only after clicking the link

Also included in his memoirs starting with “Surely you must be joking, Mr Feynman!”, well worth reading if you can get over the occasional sexist bits.

HN titles are mostly kept as titles rather than summaries/additional metadata in part for a feynmanly reason - the pleasure of finding things out.

A justification by dang some time back: <https://news.ycombinator.com/item?id=7518157>

Thanks for the link.

> It's a trick I learned from pg for keeping the focus on content rather than personalities.

IMO none of us would’ve bothered reading this if it were just some random person’s commencement address. I personally would’ve been bummed if I skipped this post just because I didn’t find the nonsense title (nonsense unless you read the article) interesting enough.

A number of people would likely have recognised the title and publication date and inferred authorship. The piece is well-known.

I don't fully agree with HN's views on authorship. It's often inferrable through domain names (which serve as something of an authorship proxy, and are often weighted internally to HN's ranking algorithm). I'd much prefer there was more metadata on articles, and remember Slashdot's practice of providing a brief summary blurb with some fondness. (Slashdot itself is sadly largely irrelevant, at least for discussion. As an article aggregator it has ... some uses.)

Reply on news.ycombinator.com