Comment by advael | Hacker News Reader

Comment by advael | Hacker News Reader

When I talk to scientists and engineers and just generally people who value improving their accuracy and reasoning ability in some capacity, I share with them the assumptions that I think can be fairly called the intellectual norms of academia, which are that strong, vague, and unqualified statements not backed by a lot of evidence are likely to be nonsense and people who make a lot of them are unserious. People can admit they don't know things, express uncertainty, make guesses, make assumptions, or have gut feelings, but it is important that these are qualified and labeled as such, and failure to do so lowers your status in this context

However, most people who make hiring decisions outside of academia (and increasingly many inside it) are part of a different culture which assigns status based on the confidence a person projects and their ability to inspire other people to feel strongly about things they say. While there are also people of the former culture making hiring decisions in sufficiently technical fields, it's hard to avoid the pressures of this second culture.

Also, the pressure on knowledge workers is the same as that on workers of any kind: While there are a few stray independently wealthy people or even quasi-independent rags-to-riches successes in the world, by the numbers most people have to dance for their dinner, so they have to either become conversant in this culture of confidence or find people they trust to navigate it for them

I actually don't think the second culture has no value. It's a lot more powerful in building organizations and gaining buy-in from lots of people. I also don't think the first culture completely avoids the failure modes mentioned in this article. Obviously people with what I've called academic norms definitely fall into this article's discussed pattern of trusting people based on their credentials and disproportionately Think They Are Very Smart

That said, since people fluent in the second culture tend to rise to the top of organizations, people in the first culture increasingly need to signal confidence even when it's inappropriate to justify their existence as they move into a professional sphere, which again most people don't really have the option to opt out of

The second culture is built for politics. There is a neat dividing line - whether questions of truth should be settled by consensus or trial against nature.

The academic STEM culture is a very carefully nurtured group of people who believe (at least to a greater extent) that truth should be decided on by a repeatable, impartial procedure that leverages natural phenomenon. So if two engineers disagree about the minimum conditions for a building to collapse, they are usually the sort of people who will converge on what happens in reality.

The more normal culture is to accept that while there may or may not be an objective truth, what really matters is what most people think. Narrative suddenly becomes important and the truth depends a lot on what you are trying to achieve. Think lawyers and judges. They may accept an objective truth exists in principle, but at the end of the day to them an argument can only be settled by most people agreeing on the truth (including dead people if tradition gets involved).

The first culture is more powerful when dealing with reality (to the shock of nobody). The second is more effective at dealing with groups and generally is more pleasant to work with, except when they lose touch with reality completely and goes berserk with groupthink.

Coincidentally, these dynamics explain why political bodies often choose to drive over the cliff rather than swerve.

> The second is more effective at dealing with groups and generally is more pleasant to work with, except when they lose touch with reality completely and goes berserk with groupthink.

The uncomfortable truth is that such people always have lost touch with reality, that makes them very easy to deal with when they like you since they will overlook your faults and mistakes. But when such people dislike you then they will only see faults and mistakes in what you do.

So it isn't that the first and second versions of those are more attuned to reality, the difference is only how positive or negative their lack of reality check is for you.

Arguably the second culture is also built for leadership. Evidence can't tell you which direction to take a company in, and the person that decides on this direction needs to project confidence in others to instill it in others.

Don't agree with your framing on the dividing line. I think the real difference is between groups that treat truth seeking as the end goal vs. those who treat truth seeking as a means to action. These two groups are in constant tension with each other. The first group will always bias towards low uncertainty but will often be paralyzed with inaction. The second group will bias towards action, but will risk choosing the wrong action. Both groups are valuable but often conflict

Knowledge-seeking versus goal-orientation honestly doesn't map one-to-one here, though it's probably correlated. Like I would say that is a better distinction between engineers and scientists than between both and good salespeople or politicians. Goal-oriented people can trust specificity more and knowledge-seeking people sometimes trust confidence. I think at the absolute extremes these map pretty well, but much of the middle is more complicated.

“The fundamental cause of the trouble is that in the modern world the stupid are cocksure while the intelligent are full of doubt.”

― Bertrand Russell

This adage has been debunked[1].

[1] https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect

Do you mean to draw our attention to the part at the end of the "Definition" section that begins Among laypeople, the Dunning–Kruger effect is often misunderstood, or do yourself understand it to mean "stupid people are too stupid to know they are stupid", in which case this doesn't contradict Bertrand Russell?

Dunning Krugers study shows that accuracy correlates with confidence, that goes against the quote. In general people who are more confident are more likely to be correct.

> In general people who are more confident are more likely to be correct.

Interesting. That is so completely at odds with my experience that I find it hard to actually believe. I guess I need to look at the research.

I think there is a different between:

1) Answering a specific question: "how confident are you in your results?" In this context, confidence is correlated with correctness.

2) Projecting external confidence in a social context. In this case, your experience and mine agree that the most overtly confident people are generally not the most competent ones.

Dunning Kruger is relevant in the context of the linked article but is not relevant in the context of the parent comment.

Dunning Kruger study shows that confidence correlates with accuracy, which goes against the quote.

Veblen (of Veblen Goods fame) described it in his Theory of Business Enterprise. Its called a Veblen Dichotomy. As soon as the chimp troupe discovers a new tool/tech/process etc, two groups emerge to handle the consequences. Both play different roles. One is more technical focused, the other resembles a ceremonial class (similar to kings court). Veblen felt with advances in tech the need for a institutional/beauracratic/ceremonial class would diminish with time.

But it hasnt happened yet cause accessing funds, managing finances, building networks, attracting talent, customers, handling competition and conflict while keeping ever growing group size (ie factions and differdnt agendas,needs,values) from breaking apart, requires lot of tactics that tech hasnt fully addressed yet.

There is also a thing that could be perhaps called "the neophobic paradox": excessive neophilia halts progress, as it turns it into a random walk. It turns into people chasing the next big thing, with no noticeable progress over longer timescales.

neophilia: love or enthusiasm for what is new or novel

Yes, and you accept anything new when you are too neophilic, and turn progress into a random walk, because most new things are crap.

Good to know it has a well-known name

A big problem I've found with academic types is belief that their intuition translates to other areas of the sciences or worse, engineering. There are many flavors of academics, but experimentalists trying to perform engineering-like work is an amazing exercise in futility. There is immense difference between finding out how something works and getting it to work in a consistent and repeatable way. I imagine the nuances across other disciplines are similar

in my experience academics are the worst at understanding uncertainty. They refuse to believe anything that doesn't pass their beloved t-test, believing the holy threshold of 0.05 is the arbiter of all truth

You are conflating the typical threshold for drugs to pass random placebo controlled trials with all of academia. That demonstrates a lack of understanding and places you firmly in group 2.

It's not just drug trials, it's a huge proportion of biology and psychology research at least. Additionally, most research in biology is expensive to get off the ground, and in order to get funding it absolutely is important to project confidence - there's an entire game surrounding grant writing that sounds a lot more like "group 2" than "group 1". Group 1 is just a theoretical ideal, not the reality of academic science. While that may be somewhat field dependent it's definitely not restricted to a small niche either.

My wife's course exercises in statistics for health studies says "if the p-value is below 0.05, the relationship is statistically significant." Sure, it's a medical institution (Karolinska institutet in Sweden), but the programme she's in is about workplace health management, not even close to drugs.

Two data points do not a generalization make. I work in engineering sciences, and we have no such threshold.

The problem I have with experts is that they are confident that they understand what someone else is articulating.

One problem is that experts use their own jargon and often confuse one use of normal words with their jargon.

The second step is dismissing rather than inquiring.

This actually strikes me as an example of a political decision creating a stickiness to academic formal standards (as others point out this has become a policy for various organizations' approval of findings). Since the "replication crisis" that happened in the last decade, a lot of people's explanations for this have centered around the use of the fisherian significance test and how gameable it can be, as well as pressures toward publishing positive results. I think there's some merit to solutions involving changing those standards and a lot of STEM people have advocated for them. However, things like drug trials have politically-determined standards that need to be upheld, and some of those standards have become well-known by the general public, so they can't really be bugfixed quickly

There is no single or set of standards that can prevent you from having to think critically about a result. Rarely are results so overwhelmingly positive that it's obvious what the result means. In those cases it doesn't matter what method you use anyway. The rest of the time, you should use many methods, be wary of their assumptions and think very hard about what the data is telling you. Realize that you are biased in favor of a positive result and play devil's advocate with yourself. This is very very hard work, and most people don't want to do it, especially when their careers improve when they look the other way

I view this as obvious but I guess that doesn't go without saying. The two cultures I've characterized actually share a commonly held strong belief that there exist metrics or heuristics that can replace the need for fluid critical thinking. Obviously it's not everyone in either culture who thinks this way, but it's a dangerous form of automation for both

I agree with you in principle but the reality is that most Universities and research organisations are extremely political. If you challenge the beliefs of the scientists in charge, your funding and support will often be cut. The history of “Nutritional science” is a great example of how bad it can get when money and politics overrules basic science.

Reply on news.ycombinator.com