Hacker News Reader

Comment by TimPC | Hacker News Reader

The article is correct. The effect is statistical not psychological. It emerges even from artificial data and occurs independently of the supposed psychological justifications even for data where those justifications are clearly removed.

If you adjust the experiment design to avoid introducing the auto-correlation you get data that doesn't show the DK effect at all. Some might take issue with the adjusted experiment as using seniority related categories like "sophomore" and "junior" as skill levels has its own issues. To show the DK effect is real you need to come up with a better adjusted experiment that avoids the autocorrelation while still generating data that generates the effect. It's unclear if that's possible.

View on HN

Just anecdotal, but my life observation of DK is often highly intelligent and competent people in a particular field who then generalize that to pontificate and proclaim, directly or indirectly, superior understanding to certified domain experts (e.g., have directly related advanced degree(s), work in the field for decades.) It thus seems more or as much a psychological effect - in short, people with a personality type of superiority and know-it-all, yet have never done the deep and hard work to gain or demonstrate any competency in said areas. A common side observation is of course unfounded conspiracy theories, that the derided experts have sinister intentions.

Classic example:

>The V-tail design gained a reputation as the "forked-tail doctor killer",[16] due to crashes by overconfident wealthy amateur pilots,[17] fatal accidents, and inflight breakups.[18] "Doctor killer" has sometimes been used to describe the conventional-tailed version, as well.

https://en.wikipedia.org/wiki/Beechcraft_Bonanza

To be fair, I think the counterargument is that ostensible experts can overstate their ability/skill/knowledge to detrimental effect just as easily, and by virtue of the label ignore the reality of the argument/scenario at hand. That is, experts can overlook mistakes they're making, or conflicts of interest, etc because of their status, and because they overestimate their own ability. There's been studies of this in group decision making in crisis situations, where hierarchies can cause failures because the "leader" becomes overconfident and fails to heed warnings by others in the group.

This all gets really murky quickly in practice because of what "low" and "high" competence means, and what constitutes the actual scope of expertise with reference to a particular scenario.

https://xkcd.com/793/

Anecdotes are not data. Even in plural form. We can witness isolated instances of what seems to be a phenomenon without it actually being part of a phenomenon. Other posters have already established that some experts can overestimate their expertise as well. The study mentioned by the original post and in my comment seems to suggest that the overestimation bias is prevalent across a wide range of cohorts of expertise. Senior students are just as likely to overestimate their talents as freshmen for instance. This effect likely extrapolates to experts as well it's just hard to get good data.

No DK doesn't say no bluster, no proclamations or no artificial assertions of expertise. It doesn't even say that the overestimates are just as prevalent among experts as laypeople. All it says is as near as we can tell the effect size of the overestimation is the statistical autocorrelation and our best efforts to produce the same effect without relying on the autocorrelation have failed.

I think there are a lot of ways to accept the anecdotes you mentioned occur that need much weaker assertions than DK as a psychological phenomenon and would hesitate to jump to DK based on that information.

> The effect is statistical not psychological.

It is, though. This article says that if people are bad at estimating their skill (towards randomness / the midpoint), then bad people will overestimate, and good people will underestimate.

The psychological part is that people indeed will assume they're closer to the mean than they actually are. DK effect would not be seen if people correctly estimated their skill, nor if experts overestimated, nor if the incompetent underestimated.

Yes you are right and I don't understand how so many commenters in this thread can be so confidently state that the article is wrong. I just went ahead and did the random simulation myself and you get the "Dunning Kruger effect" which is exactly what the author paints it as: autocorrelation.

>If you adjust the experiment design to avoid introducing the auto-correlation you get data that doesn't show the DK effect at all.

This is even in the article! Yet some people are making claims against this despite references to the contrary.

The article shows that the qualitative claim, that low scoring people tend to overestimate, and high scoring people tend to underestimate their ability, is nothing but a statistical obviety.

For the Dunning-Kruger effect to have psychological significance, you must quantify this, and show that they overestimate their abilities more resp. less than expected.

Regardless of how you set your expectation/null hypothesis, the absence of the effect would mean that the lowest scoring quartile would on average estimate their abilities to lie at 50% or below. It is found however that people in the lowest scoring quartile position themselves in the third or fourth quartile.

I'm not saying that this is necessarily deep or unexpected, just that the article only shows that the qualitative statement is true regardless of any psychological factors, and not that the Dunning-Kruger effect doesn't exist

The y-axis is actual_score - self_assessed_score, right?

So we are interested in whether the average of this y-axis value differs between high scoring and low scoring groups. The study that shows there is no D-K effect [1] linked by the OP article shows there isn't a difference in the average value (y-axis) between groups. In fact the mean value was 0 for all quartiles. So we can conclude that we have found no systematatic over/under reporting of actual vs percieved skill. People in the lowest quartile do not appear to position themselves higher than they should. Or at least, we can't reject the null hypothesis

1. See figure 2 on page 15, note that zero means agreement between self-assessed and actual: https://digitalcommons.usf.edu/cgi/viewcontent.cgi?article=1...

Reply on news.ycombinator.com