Hacker News Reader

Comment by kelseydh | Hacker News Reader

Max Planck argued that change takes time because good ideas need enough staying power to outlive their detractors:

> A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die and a new generation grows up that is familiar with it … An important scientific innovation rarely makes its way by gradually winning over and converting its opponents: it rarely happens that Saul becomes Paul. What does happen is that its opponents gradually die out, and that the growing generation is familiarized with the ideas from the beginning: another instance of the fact that the future lies with the youth.

https://www.benwhite.com/misc/good-ideas-need-to-outlive-the...

View on HN

I suspect that there are myriad counterexamples of this belief, and that it's not even testable. Perhaps one idea that he was involved in -- quantum mechanics -- encountered resistance because it had glaring problems that were not resolved with one killer test or theory, but the problems were gradually overshadowed by the empirical success of the program.

Please don't turn this -- or any hypothesis -- into a "law" of how science works.

Based on our understanding of human psychology, despite it being as incomplete as it is, it would seem reasonable to argue that this trend, if it does exist, would not be a binary rule but one whose strength depends upon numerous factors that coincide with how strong the evidence is, how many different experiments create evidence that aligns with the new theory, and how much better the new theory explains existing problems in the data.

If the existing theory predicts results that are 50% off, and the new theory is 45% off in the other direction, then things aren't likely to be accepted. If it is instead 0.1% off, that makes a much stronger argument. The issue with rejecting the first case outright is that often the experiments themselves have imperfections, but those are much slower to work out and refine. When the new theory doesn't need the existing experiments to be refined, I would guess long term experts are much more likely to entertain it.

> depends upon numerous factors that coincide with how strong the evidence is

Might I venture the guess that seeing tangible advantages for oneself when using that new theory is more important? If you adopt it, will you become one or more of more famous, more successful, able to advance your own work, become part of a more reputable group, etc.?

If it is merely something that will not affect you, there is little or no incentive to change one's view.

It's not a law. If human civilization goes extinct, no more "scientific truths" will be discovered. Or an authoritarian society could gain total control (perhaps with robotics), and for whatever reason, they decide to ban some fact and mandate that everyone be taught the lie.

But scientific truths have a tendency to be adopted that falsehoods don't because truths are verified by other truths, whereas falsehoods contradict them. A lie can only be so large before it becomes self-contradictory, the truth is incomprehensibly large yet coherent.

If quantum mechanics has problems, as we do more experiments, we'll encounter these problems more and more, until eventually they can no longer be overshadowed. I predict quantum mechanics will never be entirely thrown out, but it will end up as a simplified approximation of a more complex "true" model; which is still taught in schools and used wherever extra accuracy isn't needed, like classical mechanics is today.

I think Max Planck's quote matters in practice too. In theory, you can discover a "scientific truth" and be recognized for it, but only long after your death, and only after someone re-discovered it (i.e. you didn't advance scientific knowledge at all).

However, most people aren't particularly unique, which means that if you discover something, chances are others have discovered it, or are at least close enough that you can easily convince them. You may not be able to convince the dominant "in-group", but if your idea is obvious enough (which it probably is if it's true and you managed to discover it), you can form an "out-group", which will grow as the idea gets verified (by truths) far more often than contradicted (by lies, because there are less of the latter, since a lie can only be so large without being self-contradictory).

Why do science in the first place? If your only goal is to predict something, you're doing it for yourself, so do "Science 1" and listen to others, but only to correct yourself. If your only goal is status, the truth doesn't matter to you, so do "Science 2" and make others happy. If your goal is to further scientific knowledge, I recommend you do both with preference for "Science 1": prioritize being correct, but explain your idea very well and make others happy when possible without sacrificing correctness (diplomacy).

It's important to note that when you can't change the majority's factual belief, you should really evaluate your own, because usually in such cases you're the one whose wrong. But otherwise: when you can't change others beliefs, the next best thing is to (as much as possible) not care what they believe, even if they are the majority.

>If quantum mechanics has problems, as we do more experiments, we'll encounter these problems more and more, u

This is non sequitur. If you do more experiments, you'll encounter it. But if there were such a wrong theory that everyone insisted was correct, it's not necessarily the case that more experiments will occur. I can't do an experiment to invalidate... quantum mechanics is in a regime where anything less than hundreds of thousands of dollars does not even get you started (never mind the lacking expertise). It's also unlikely to be financially lucrative in the timespans that would entice an investor (especially one averse to pissing off the status quo, as most are). By doling out the grant money (or not) to those people who will preserve the status quo, one could let the false theory survive for decades or even centuries.

We seem to have this mythology of science from a past era, where some maverick can just bust in and start embarrassing people with unignorable truths. If ever there was such a time it exists no more. The stakes have never been lower, trillions won't be lost if we get quantum mechanics wrong. Thousands won't die in a quantum mechanics accident that could have been avoided. The problems could persist indefinitely.

It's true that a false theory can survive indefinitely, especially if it doesn't have real-world impact. Classical physics survived for centuries before being disproven by modern physics, because the differences are subtle except in extreme circumstances. Maybe quantum physics is accurate enough that the circumstances for non-negligible differences between it and "the truth" are so extreme, we never test them, therefore it's never disproven.

However, right now people are spending large amounts of money to build quantum chips. They're not explicitly trying to disprove quantum physics, but implicitly testing it through their experiments. And if these tests suggest that quantum physics has fundamental issues, they'll investigate, at least so if they realize quantum chips are impossible, they stop spending money trying to build them.

Every transistor in every existing computer is designed on the basis of quantum mechanics because they're dealing with electrons and atoms.

Quantum physics is too basic for any experiment to not test it. Like the other poster said, all the truths are linked together in physics and math especially.

> If quantum mechanics has problems, as we do more experiments, we'll encounter these problems more and more, until eventually they can no longer be overshadowed.

The problem is rather that a lot of physicists tend to "hand-wave away" that existing problems (like "what is a measurement?" and "sudden collapse of the wave function") actually are problems in the theory, i.e. that we don't have "the theory is basically correct (as evidenced by lots of experiments that were done), but these 'problems' are simply open, unanswered research questions".

They're problems when they matter experimentally. If quantum computers are failing because their qubits keep getting measured, the companies building them will spend a lot of money to discover why, which will refine our definition of "measurement" at least in that context.

> They're problems when they matter experimentally.

This is exactly an example of the "hand-waving problems away" point that I made. :-)

They should be hand-waved away.

I'm an industrial physicist. I've noticed that physicists, and the general public, often have different ideas about what problems we should be versed in. And people are surprised when they learn that most physicists are not theoreticians. Were it up to the public, we'd all be working on warp drive, infinite energy, and explaining quantum mechanics. ;-)

We all learned about the problems in both undergraduate and graduate training, and in discussions and readings. I attended a lecture about it by John Bell. I was excited about it, but I also had an experiment to finish.

I think physics is utterly unique in having a theory with seemingly infallible predictive power and zero explanatory power. But if someone asks me about it in the lunch room, all I can do is shrug it off. The fact that this paradox hasn't stopped us dead in our tracks, in 100 years, that's the problem.

> I'm an industrial physicist. I've noticed that physicists, and the general public, often have different ideas about what problems we should be versed in.

Rather say: you have studied physics, but what you actually work on and are interested in is engineering. :-)

Addendum: Just to be very clear: there is nothing wrong with being exciting about engineering problems from industry.

Actually, I'm a scientist, not an engineer. I don't want to be an engineer. I realize we're outnumbered by engineers, and many people have never met a physicist. My parents were both industrial scientists too. We exist. There's a general sense that while there's some overlap, scientists and engineers are not the same, and may even think differently.

This tracks perfectly with Khun's Structure of Scientific Revolutions. The paradigm only shifts when the old guard dies off. Scientific progress has much more to do with ego, institutions, and human nature than scientific method.

https://en.wikipedia.org/wiki/The_Structure_of_Scientific_Re...

Tends to be the issue with a lot of human society. Although, reading through the summary on Wikipedia, I'd suggest that a lot of the "paradigm" shifting often has to do with humanity's monopolistic rent extraction / tragedy of the commons nature.

Almost every part of society, including science tends to collapse towards those that have access to ideas and resources, and are placed within society to be able to take advantage of them, and those that are exploited. Even the Nobel Prize was found to mostly favor those who were born to win a Nobel Prize. Best way to win a Nobel Prize? Be born to a wealthy family with a prestigious science background.

Those that have access tend to devolve into monopolistic tendencies, exploiting the existing "paradigm" for their own gain, while punishing, excluding, or minimizing those that suggest alternatives that might in some way disrupt their power structure and control. Money, politics, military, industry, and a lot of other parts of society seem to all work about the same. Best way to work at the White House? Be born to a wealthy family with political access.

Almost every person who has retrospectively been considered "great" or "revolutionary" centuries or millennia later were punished, excluded, or minimized during their own lifetime, while their contributions were then reevaluated, fought over, fetishized, and collected after their death. Galileo was mentioned elsewhere, relatively well known example, some success and acceptance at the time, mostly condemnation for "vehement suspicion of heresy." Van Gogh's work only began to attract attention in the last year of his life ... right before he shot himself in the chest with a revolver from being so mentally tortured.

Why wait when we could just not be dicks?

If your solution to some problem relies on “If everyone would just...” then you do not have a solution. Everyone is not going to just. At no time in the history of the universe has everyone just, and they’re not going to start now.

— <https://squareallworthy.tumblr.com/post/163790039847/everyon...>

Did you just

It can be part of a viable solution.

Someone’s plan when investing in early Solar panel R&D went something like: If everyone would just… “follow their economic interests” driving down the cost of panels will dramatically increase adoption further driving down costs in a feedback loop.

Unlike most “if everyone would just” plans that one actually worked because the desired behavior aligned with people’s interests.

> Someone’s plan when investing in early Solar panel R&D went something like: If everyone would just… “follow their economic interests” driving down the cost of panels will dramatically increase adoption further driving down costs in a feedback loop.

I'm not sure if that is a good paralel. The difference is that we didn't needed "everyone" to innovate on solar panels. It was enough if "someone" was, and those who did not got left behind with their inefficient processes. That's not a true "everyone would just" situation.

Getting a handful of people to act differently is rarely the issue, especially if they think they’ll get rich by doing so.

The customer base continually expanding is the “tough” side of the equation. 100’s of millions of people behaving differently is the hard part of those “If everyone would just” plans.

Its not just a tendency to assume the best or worst, instead of investigating and dealing with reality. Its a full blown feature of evolution:

Its OP. Signals engagement and understanding to society, while expending no energy on work. Idealisation is not retardation, its optimisation.

You need both people with their heads in the clouds dreaming of fantasy futures as you also need those promoting the status quo. The former gives us direction and hope (motivation) while the latter gives us stability. There is a balance. To use reinforcement learning terms: exploration and exploitation. In reality there are many more subsets that each pull in different directions. I agree that this is optimization and most of them play essential roles. But it also means we should adjust the weights and pay attention when one starts to dominate and throw things out of balance. And I'd argue that this is exactly what has happened in academia. The bureaucrats won and threw things out of balance. I'm not asking to go somewhere we've never been before, but I think many have because they don't know what the past looked like.

  > “If everyone would just...” then you do not have a solution

Unfortunately this is not a reasonable argument. I get where you're coming from but what I'm asking is that everyone just do their job. Surely "do your job" has to be a reasonable version of this.

What I mean by "not be a dick" is to check the alignment, the goals of the process compared to what we're actually achieving. What is the point? The author of the article lays out a lot of reasons and even is stating how these things are well known. Which unfortunately means someone needs to actually take action. When we're in a situation where many people want change but no one is willing to fight for that change, then we will just keep doing what we've been doing and headed where we've been headed. Even if that is knowingly off a cliff.

I don't need everyone to just do something, I only need a few more people to stand up. And yes, I will tell those that are saying "keep your head down" to shut up. Some things are worth fighting for and for me, one of those things is the integrity of science.

>[W]hat I'm asking is that everyone just do their job.

>I don't need everyone to just do something.

Naked contradiction. Either everyone needs to just do their job or not everyone needs to just do their job.

>Surely "do your job" has to be a reasonable version of this.

There are entire fields of research centered around answering why people don't 'just' do their jobs like good little worker bees in exquisite detail. Some terms I'm aware of that you may find useful to look into, in rough order of how general to the problem they are: Agency problems; the Case theorem; malicious compliance; work to rule; collective bargaining; moral hazard; perverse incentives; adverse selection; rent seeking; regulatory capture. If you want to read up on people trying to design actually working systems from scratch, look into the world of mechanism design, starting with auctions and branching outwards.

>When we're in a situation where many people want change but no one is willing to fight for that change, then we will just keep doing what we've been doing [...]

One could argue that the past ~century of scientific and technological development has probably beat any other 100 year period your could pick hands down along any natural metric. So "what we've been doing" is actually pretty great, and it may not be a good idea to stake such a hugely important enterprise on some newfangled and only theoretical ways of doing things.

> >[W]hat I'm asking is that everyone just do their job.

> >I don't need everyone to just do something.

> Naked contradiction. Either everyone needs to just do their job or not everyone needs to just do their job.

It's not a contradiction; "just something" is not the same thing as "their job". "I need everyone to do their job" does not contradict "I don't need everyone to just do something." (Emphasis added for clarity about the differences.)

  > One could argue that the past ~century of scientific and technological development has probably beat any other 100 year period

You could make this argument about most centuries. But it's a meaningless argument if the metrics you're evaluating on are implicit and assumed to be well agreed upon by all others.

My reply is the same to the other arguments you've made

Almost no one is a dick on purpose. Everyone belives in their truths. Mutually incompatible truths. That is perceived as being dick from other person's truth perspective. That's the tragedy of situation. Everybody is right to some extent. Plus ego plus interest plus strong belives makes hard to move from own truth. So no, we can't stop being dicks. We're just being normal humans.

It would be nice if that was true. But school and university has taught people to first and foremost be obedient. That means a large majority of people doesn't care about the truth at all, only what the relevant authorities are saying.

Many of those authorities have learnt that deciding what is true based on reality is good and all, but you live longer and better by making friends and not disagreeing with them.

This is a societal failing.

This is such a weird perspective because the attitude I associate with people who go through the university system on the academic path is not obedience at all, but ruthless self advancement. Like literally where are all these obedient people you are talking about?

I would characterize the problem with science as being a failure to increase the available resources commensurate with the population of people capable of doing science. In this situation, the competition becomes sufficiently fierce that it is statistically better to lie, cheat or knee-cap your competitors in some other way than it is to actually do good science, which is unreliable. What you see as fealty to scientific authority is actually just a system which has become totally dominated by resource competition to the exclusion of its actual purpose.

> school and university has taught people to first and foremost be obedient.

That's not inevitable. I count myself lucky that it didn't happen to me.

Unfortunately I don't know how to improve schools to the level of those I attended over half a century ago. And I lack the get up and go to make it happen anyway.

  > Almost no one is a dick on purpose.

This is true, but that does not mean people are not being dicks.

What is important is to have self reflection and to recognize when you have been a dick, to apologize (make amends if necessary), and try not to repeat. Yes, habits die hard, but we can still improve. But the biggest dick move is to double down. We've created a culture where we act as if being unintentionally wrong is a bad thing and that the worst thing you can do is admit a mistake and self-correct. But we are always wrong (to some degree) and so the only thing there is is to self-correct.

So yes, we can stop being dicks. It's how humans evolved. We wouldn't have expanded from our small tribes to villages, to cities, to countries, to a global economy if we weren't capable of this. The arc may be slow and noisy, but it has always expanded to be more inclusive.

Because you don’t know which side is the “dicks”. Evidence is rarely conclusive, and progress isn’t made by cowtowing to the researcher with the biggest mouth.

Why risk it when you can be a dick and assure your success?

If you are using a first order approximation, then yes, this is the correct strategy. But when you consider the elements of time or consider that there are other players in the field (incorporating either will do, both is better), then this strategy becomes far from optimal. In fact, it will lead you down the wrong direction, making your own life harder. The thing is, bullshit compounds. Builds up slowly, but we all know that's how you boil a frog.

Humans have always advanced through the formation of coalitions. To optimize your own success you have to simultaneously optimize the success of others.

There is right now a thing that you believe strongly that is false. If someone were to point it out to you you would get angry. C'est la vie.

Yes. But you have to convince me that it is false (of course)[0]. I'm actually happy to tell you what I think the best way to do that would be. Would I get angry? Probably not, but it is hard to know.

In fact, this is even how I review papers. I am much more detailed than my peers, and get very specific. I always also include a list at the end detailing what factors are the most important and what I think the authors could do to change my mind (if I'm rejecting). If I'm accepting, I'll also argue my points to the other reviewers and make them stand for their arguments.

Truthfully, if no one is willing to change their minds, I'd say they can't be a scientist. It is a fundamental requirement simply because we are all wrong and all the time. While we can get ever and ever closer to it, absolute truth is fundamentally unobtainable. So you must always be able to update your beliefs, or else you will become more wrong as time marches on.

[0] I also recognize that the inability to convince me does not mean I am right and the other person is wrong. But this too is why I specifically make a point to try to help the other person. At least as long as I believe they are acting in good faith. If I am wrong then I WANT to know. I take no shame in being wrong, but I take a lot of shame in being unwilling to right myself.

I actually really want to test your theory on myself. . . I wonder how I could best do that.

My method is to help your "adversary". The way I think about it is this: we can't obtain absolute truth, so we're always somewhat wrong; we have limited data and information, so we need to be able to consider what others have that we don't. Arguments can be both adversarial and cooperative, right?

If your goal is to seek truth, then you need to reframe the setting. It is not "I defend my position and they make their case", that is allowing yourself to change but framed to maintain your current belief. Sure, you have good reason to maintain your belief and I'm not saying you shouldn't hold this, but it should be a byproduct of seeking truth rather than the premise.

Just pick a position you feel strongly about and imagine how your world would change if it was false. How your relationships would change. How stupid your previous statements would be.

Pick anything. Climate change is a big one. I would definitely have to eat some chaff if it was shown to be false personally.

Reply on news.ycombinator.com