I think Dunning Krueger makes intuitive sense. When you become skilled in your field you learn from other people in your field, and your assessment of yourself is based on your relation to the skills of those other people. But if you know very little about something, you have no reference point to evaluate yourself against.
When you learn something you also learn what are some of the mistakes you can make. You evaluate your performance then against the mistakes you didn't make. Consider a piano player, or figure-skater. You have to know about what figures are difficult to perform to evaluate a performance, and you don't know what the difficult ones are until you have studied and tried to perform them.
It’s been argued before that this is the only reason that DK gained any notoriety; because it feels right, not because it is right. It’s the “just-world” theory: we want to believe that confident people are overcompensating.
Is it actually intuitive though? Consider your own example. Most people who don’t know piano or figure skating are well aware that they don’t know, and do not rate themselves highly at all. Would it be surprising to learn that people who don’t know any law or engineering don’t often hold any doubts about their lack of skill, and by and large are not deluded nor erroneously believe they’re great at these things they don’t know?
The DK paper didn’t measure knowledge-based skills like piano, figure skating, or law. It measured things like the ability to get a joke, and conversational grammar. How would you rate your own ability to get a joke? (Does this question really even make a lot of sense?)
It’s important that the methods in the DK paper focused on tasks that are hard to self-evaluate, because when people have tried to replicate DK with more well defined knowledge-based activities, they have often demonstrated the complete opposite effect, that there is widespread impostor syndrome, and skilled people underestimate themselves.
"Most people who don’t know piano or figure skating are well aware that they don’t know, and do not rate themselves highly at all. "
I think this case (real_skill = 0, perceived_skill = 0) is maybe a trivial case, and that the bit of truth DK-idea catches is when someone with very little skill considers how much work it would be to get to whatever a 'fully skilled' version would be, they woefully underestimate.
Picture someone in their first summer of mountain biking watching youtube videos of the best guys. Yes, you know you can't jump like they do, or turn as skillfully, but you're getting better each month. However, you still grossly underestimate how hard it is to get to that skill level.
At least it's my personal experience as a thoroughly unskilled!
Notoriety means “the state of being famous”, which is the meaning that I intended. I actually don’t want to use “notable” in this case, because that would imply that I believe the DK effect is real. Notorious might even be a good word here, since the paper has problems with its claims and its interpretation of it’s own data.
Thinking about all the "INAL" answers and the obviously wrong "legal" advice and opinion on legal topics you can come across on HN on a daily basis, I think law is good example of people overestimating their knowledge.
Replace figure skating with any other sports, so, and try having discussions about, e.g, a defeat of any soccer team. All of a sudden everyone just became a soccer coach. And everyone is able to critique individual player's performance and skill and technique. If anything, this proofs the DK effect rather well.
You might be forgetting that DK demonstrates a positive correlation between confidence and skill. They gave statistical evidence for people who believe they’re right actually being right more often on average. The question the paper is actually asking is why aren’t people’s self-estimates perfect, but it does not, contrary to popular misunderstanding, demonstrate that confident people are lower skilled. Reading bad legal advice on HN is not a demonstration of the so-called DK effect.
But is DK not explicitly about assessing yourself and not about assessing others?
I feel like being able to critique the performance of others is different from being able to critique yourself. There might be a correlation between the two, but they're not the same.
The DK paper is not about assessing one’s self. There was a self-eval, but the primary methodology used for most of the data & conclusions was to rank one’s self against the others in the group! You are spot on -- this ranking is a major problem for the credibility of the paper’s narrative. Being unable to rank against others precisely, especially when you don’t know their skill level, does not demonstrate that someone is unaware of their own lack of skill.
I think it's actually the next step; to get good at something you have to get good practice, and that requires good self-assessments and knowing how to practice the part you're weak on.
There are people who can't advance because they can't see the problem, and people who can't advance because they can't (or don't want to) correct it. The end result is the same though.
It's also intuitive if you think about error in self assessment. Skill is asymptotic to some upper bound. The closer to the asymptote (higher skill), then most likely the error in estimation is under it, since it cannot be above it.
Conversely it cannot also be under zero, so error is most likely going to be above the actual skill line (over estimation, since it's clamped below it).
Most people can't play the piano or skate. Don't consider those ones. Consider the things that everyone can do. Let's pick driving a car. I am fairly convinced that most people feel after a few years they are excellent at driving their car but in fact they are just OK to terrible. And this is with a lot of practice!
I think that's assuming more ignorance than even an unskilled person has.
If you had never listened to a professional play piano before then you'd have no idea what level of performance is possible. Similarly, if you had never seen skilled skaters perform on TV.
But we have done these things, so it's obvious that they're doing something that's very difficult.
Maybe you don't fully appreciate the skill, though. You wouldn't do well as a judge who compares the performances of professionals. But comparing novices to professionals seems easy?
If you had never listened to a professional play piano before then you'd have no idea what level of performance is possible. Similarly, if you had never seen skilled skaters perform on TV.
But we have done these things, so it's obvious that they're doing something that's very difficult.
Sometimes the things we find most impressive, in a demonstration of a skill we don't have, aren't the most difficult things.
I remember being absolutely blown away by some aerial circus tricks and stunts I saw at shows. Later, I started studying and eventually performing myself, and it's often the case that the most crowd-pleasing stunts are some of the easiest to perform.
As a performer, you could always tell which members of the audience knew their stuff, because they'd be the only ones applauding the tricks that might not have looked so spectacular, but were actually the most difficult.
It's more like intermediate (most vulnerable to the DK-effect) to advanced (utmost appreciation for professionals).
Taking the piano example: after 1-2 years of progressive learning you can certainly give off the impression to somebody unfamiliar/untrained (including yourself to an extent) that you are actually quite good: Intermediate stage.
But after awhile when confronted with more and more challenging stuff, by discovering different styles and finetuning your hearing; you at some point reach the very visceral and uncanny sensation of the countless possible roads you can now explore: advanced stage.
>Similarly, if you had never seen skilled skaters perform on TV.
then, as a person who has lived in the world and has the normal physical skills of such you probably think "whoa, how in the heck did they do that" when you finally see it.
The OP article mentions in their rumination that there's some difficulty in generalising DK:
"And maybe there’s no contradiction - there’s always room for nuance, for finding out where the Dunning-Kruger effect is relevant and where it’s not. That can be done with more studies, but only if the authors manage to agree on assumptions and basic statistical practice."
Your post reminded me of one of my favorite Adam Savage videos where he touches upon this idea you're exploring. I encourage folks to see it, he articulates it so well.
I linked to the start of the video where he begins to build the idea. TLDR is he mentions Monet painting Impression Sunrise and how it was something that people have never seen before and it took a bit of time for it to blow people away--they needed to develop "new eyes" to see the genius. Adam then dives into this idea of "new eyes". I'm sure many of us have experienced this in our life and it was so nice to hear Adam unpack it.
This rings so true! I have no idea about programming, in Supply Chain and logistics I do see the same thing so. And the most frustrating people to work with are those stuck at > 2. without realizing it.
Fascinating! I don’t know about supply chain, but I can tell you it’s the same in programming. I suspect these points apply to creative ideas in general.
Well, so far I've seen it SCM (professionally), you see it in sports basically every weekend (we have close to 80 million national soccer coaches in Germany), I've seen it in boxing (I'm nowhere near to number 2 there anymore, but usually novices come with no idea, then they think are good until someone shows them "nah, you still don't know how to box")... So I guess it is the same everywhere in any domain.
If human cultures can be characterized as default arrogant or default humble then it stands to reason that arrogant cultures will have a DK effect, and in humble cultures you won't.
I think you have the common misconception about the DK effect, which is incorrectly summarised as "unskilled and unaware".
There is also the other end of the scale where "skilled and unaware" occurs: people under-assessing their skill (presumed that this is due to judging that most people also have similarly high skill levels).
I think your two "cultures" would shift the self-assessment line up or down on the graph (constant), but not affect the slope very much (multiplier). The line shape or line slope must change somewhat since values are limited (between 0 and 100).
Even people conditioned to be humble could have a strong motivation to believe something is true and overestimate their own knowledge/ability in order to stand on what they perceive as evidence. For example a person's depression, religious beliefs, or an over-emphasized belief in DK itself could be a possible reason they have an erroneously deflated opinion of themselves, and simultaneously employ inflated confidence in irrational arguments that demonstrate why they are almost completely worthless at their field. That's pretty much how depression is secretly prideful in a sense: over-estimating our own mental ability to assess our helplessness.
When I did some cognitive behaviour therapy, I un-learned things like "all or nothing thinking" and the expectation that I could accurately predict the outcome of any course of action by modeling future performance off of a past failure.
> I un-learned things like "all or nothing thinking" and the expectation that I could accurately predict the outcome of any course of action
Do you know any words, stereotypes, or clichés for this? Or even what the related mental disorder is called if it were to become debilitating? Or a specific word for the complete clustering of related signals?
I am guessing those issues plus there related issues (¿syndromic?) are common - but I don’t know where to group it in my own mind.
It is closely associated with having a highly systematizing mind. People with ASD get drawn and pushed down a particular life history corridor. There are rewards of parental/teacher approval for high-performance in an area of profound interest, and a punishment in thef orm of peer bullying for low social skills. This conditions them to operate this way to avoid bullying and optimize for time seemingly well-spent with these impersonal systems. To justify one's own existence, there is this urge to live in a world where a narrowly focused mind is able to predictably produce an ideal world through expertise (all), and a tendency towards refusal to live outside of that, sometimes advancing into self destructive behaviour should that not be an available outcome (nothing). Getting ALL is not only about having things one wants, it is also about seeing the system work, and about identity, a sense of vindication.
Collective layer:
We see so much of it I think even among neurotypicals because we live in an extremely systematized world. Every single aspect of our world is seen by the "haves" of our society as a candidate for profitable separation through a digital layer. Anyone can end up metastasizing a systematizing mind. They just need to exhibit a hyper-focus on something impersonal and complex. As a global civilization, we have been doing this to ourselves and strapping others into it as much as we can. Mostly, only those living in rural parts of materially impoverished nations are spared this temptation.
Mental Symbolism layer:
It is symbolically speaking one entrance into a realm of mental death. With the devotion to lifeless systems the human is de-personalized, atomized, depressed, unrelational. They leave unfulfilled the inescapable truth of what it means to be human. To live your entire life this way is to betray your parents, ancestors, any friends or lovers you ever had, and anyone you could have helped.
Spiritual layer:
The pattern is quite literally satanic. The person has chosen to reign in this hell (both all and nothing), rather than serve in heaven (the humble, narrow middle path). The most powerful and beautiful of all created entities, with an astonishingly powerful mind, insisting on dwelling in a state of supreme perfection betrays the Father to become an engine of extinction.
No. I was just curious, and seeking understanding. I have friends that have told me their detailed future plans with exact timeframes and no contingency. I have seen others struggle to rationalise unpredicted forcing events in their lives - especially negative emotions when other people do not act according to their plan.
When you learn something you also learn what are some of the mistakes you can make. You evaluate your performance then against the mistakes you didn't make. Consider a piano player, or figure-skater. You have to know about what figures are difficult to perform to evaluate a performance, and you don't know what the difficult ones are until you have studied and tried to perform them.