It's also acknowledging that our thinking is partly shaped by our terms. If you think in terms of Java, you might not see an elegant functional solution. If you think in Haskell, you might not see an elegant imperative solution.
The known unknowns are like a door you haven't opened. The unknown unknown is not even realizing there's a door. It's a thrill to discover such doors.
It makes me hopeful to think that just a couple of steps off the beaten track, there are miracles that the mainstream has not dreamed of (nor me). There are more things in heaven and earth, Horatio,/Than are dreamt of in your philosophy. Such is the motivation of a would-be scientist/inventor/entrepreneur.
One may laugh at such incompetent people as in the article... but how do we know we aren't incompetent? One way is objective tests: If we can pass a maths test, we likely know some maths. If our programs behaves as we predict, we likely know some coding. Another way is to consult with a community of people reputed to be competent, and ask them if we are.
Universities do this - notwithstanding the study in the article - hopefully at least at the postgraduate level. I may be stupid, but at least I'm state-of-the-art-stupid. Of course, there are famous examples of radical scientists, scorned by the scientific establishment, who turned out to be right... (though perhaps famous more due to their appeal than their frequency.)
I'd always wondered if maybe thinking in Japanese or German allows Japanese or German people to come up with better solutions to some computing or engineering problems. Thanks for sharing that link.
I worked at an open-pit copper mine in New Mexico one summer, part of which included a pretty intense 3-day safety course. One of the best take-aways was their definition of four levels of competence:
Unconscious Incompetence: you don't know what you don't know
Conscious Incompetence: you begin to learn what you don't know
Conscious Competence: you learn your task, but need to concentrate on it
Unconscious Competence: you can perform your task without that level of concentration
The obvious most dangerous one is unconscious incompetence, but the other dangerous one is conscious competence. As you learn your task, you may become overconfident at this point, which may lead to lose of concentration on the task if you know your task better than you do.
Applies not only to working with heavy machinery at a mine, but server administration, coding, etc.
Just a few weeks ago, my five-month-old son suddenly discovered that he had hands. He's been staring at them, intentionally grasping objects, waving and gesturing quite a bit since them. Thought of another way, just a month ago, any solution to any problem that required the use of hands was an "unknown unknown" to my son.
It's been amazing to watch my son as he discovers things most of us take for granted, as he changes "unknown unknowns" into "known unknowns" or even "known".
I see this in the Irish trad and Old-Time scenes all the time. A lot of sessions have the one sweet guy or gal who everybody likes, but who is so bad, it's an empirically measurable fact. There's one Houston drummer whose technique for holding her stick is so bad, her 2nd beat each measure falls at random every time. You could take a recording in Audacity, put in the tic marks, and it would be glaringly obvious. I've encountered many musicians with no rhythmic sense and others with no sense of pitch to speak of. I've also encountered musicians who are very competent in one genre, but have not enough clue to know they're clueless in Irish trad. This weekend, I was playing with a wonderfully competent old time player who had no clue the emphasized beats in Irish trad fall on the downbeat and not on every upbeat. Doing the latter obliterates the occasional deviation that can add playful (purposeful) changes to the rhythm. It's like showing a Picasso blue period painting under a blue floodlight.
The lesson here: One can be very good in one area, but still very clueless in another, even a closely related one!
It is a wise man who knows that he knows nothing. - Not sure who really said it.
What I find interesting, and somewhat disheartening, is that the people who are incompetent and constantly brag about themselves, actually tend to be more successful because people admire confidence.
Having spent a lot of time on both the selling and buying side of "professional services" I can see that most successful sales pitches are built around giving the purchaser a sense of certainty of the effectiveness of the proposed solution.
I actually think that in many cases it is more difficult for someone who is knowledgeable to convey that sense of certainty because in so many technical situations the technical answer to simple questions is "it depends on the details of what you want to do". The "right" answer in a sales context is usually "yes, this will solve your problems" and worry about the reality once the contract is signed.
I think to some extent individuals that continue the process of learning into adulthood naturally become adverse to stating anything in completely concrete terms. Presumably part of this is from painful past experience, but also as one's depth of knowledge grows so does the realisation of just how much there is to know - even in a supposedly finite domain!
Yes, but be careful with this line of thought - it is a result of a person being aware that there are known-unknowns. That is why they avoid using concrete terms, because they know that they are ignorant about many things. But there are still things which this person will be unaware that they don't know.
I have always felt that Richard Feynmann's greatness devolved largely from the fact that he was better at identifying things that he didn't know than his colleagues. Where one person might develop a theory that works to explain a particular phenomenon, Feynmann would understand that this answer would just reveal new questions, so he would dig further than others.
It's been shown in studies that athletes who think they are the best (deceiving themselves) perform better than athletes who are more realistic. The phrase "ignorance is bliss" does indeed ring true.
I think this illustrates the sorry state of education. I learned about unknown unknowns in high school from Socrates. I hope my son learns them even sooner. There's an opportunity cost in false confidence. Better to get that out of the way at a young age, and learn to feign confidence when necessary.
But that isn't everything. At a university you start to see that subjects that might have appeared to be shallow at first glance are really quite vast and complex. It's hard to say with confidence you are an expert on any of those subjects after seeing a little ways down their respective rabbit holes (which you rarely get the opportunity to do in high school).
I don't think they get ahead because of admiration.
It's just that skill can only be judged by people with equal or more skill, if you have to judge somebody without the necessary skill to do so you have two options.
1. admit you don't know and get somebody else to judge him (admit your own incompetence)
2. judge by other factors (including confidence)
option 2 is by far the easiest and thus the most chosen one.
Dunning wondered whether it was possible to measure one’s self-assessed level of competence against something a little more objective — say, actual competence.
A much more interesting question, one for my next blog entry, is the opposite: in a complex world of millions of different specialties about which much is known at great depth, might it be impossible for anybody to know when one is exceeding one's competence? Might this, instead of being a humorous story about bank-robbers or a story about "others", be a story about the entire population of the planet?
The easy question is: given a specific field, how do we help people self-rate? But it's not a very practical question, because in the real world it's never just one vertical field to rate inside of. The tougher question is the matrixed-skills one. The brain surgeon who speaks about car maintenance as an expert although he is completely wrong, the psychologist that waxes on about social ills as an expert although he is completely off-base, the college professor in mathematics that forays into economics with nothing more than a lunch pail and a bologna sandwich. This is the really the more interesting (and important) scenario. (The hacker that ventures into cognitive psychology. Yes, I get the irony.)
Note that I picked skills that were far apart from each other: brain surgery and automotive repair, for instance. These are the easy cases. The crazy hard ones are where the skillsets are very close to each other. Not sure that outsiders could spot that happening.
There are actually two different questions about competence:
Competence compared to some objective standard, basically "Can I do that?", which I think often can be answered realistically (at least where "that" is specific enough).
Competence compared to others, or how do my abilities rank compared to others, which may not really be answerable, except very generally and vaguely.
The big problem comes in trying to generalize from specific competencies to "general competence".
The Dunning-Kruger effect is an overcomplicated explanation for a much simpler phenomenon: no one can effectively estimate their own competence. Highly competent people are also a poor judge of their own competence.
That's what the paper suggests, I should have been more clear in my statement. Neither incompetent, highly competent, nor people in between can effectively judge competence.
I think your summary doesn't include the most interesting bit imho: the sign of the error is different (incompetent overestimate their competence, the competent underestimate).
I think that bias makes sense: if you have 0% competence it's hard to underestimate, and if you're at 100% competence it's hard to overestimate. So if we assume that they're truncated normal curves and translate them into a simple average, that's what we'll get.
In the Bible, Proverbs incessantly mocks those who lack wisdom, who are simply called fools. Like an illustration that fools instinctively repeat their mistakes like a dog comes back to lick up his own vomit. Nice.
Then comes this thought: "Do you see a man who is wise in his own eyes? There is more hope for a fool than for him."
In some of the cases, the word translated 'fool' is the Hebrew word nabal which should better be translated as 'perverse.' Thus the verses which says, "The fool hath said in his heart there is no God, They are corrupt, they have done abominable works, there is none that doeth good." (Psalm 14:1; Psalm 53:1)
"Unknown unknown solutions haunt the mediocre without their knowledge."
Discovering new solutions and ideas often involves a lot of naivety - Making progress often requires a willingness to approach a problem with a truly fresh perspective, free of the prejudices and blindness that come from having too much depth in a field.
While I can't always find the discipline for this - I like to try and attempt a problem on my own, even when I strongly suspect that there is an existing solution better than anything I could invent. I am surely less competent than someone who knows the existing solution and applies it without hesitation, but the more competent expert is less likely to improve his methods unless he is willing to move beyond established methods where he risks screwing up.
Intelligence can emerge from stupidity.
This is actually a heuristic for solving the reinforcement learning problem: there is a known tradeoff between exploration and exploitation. If the vast bulk of the solution space is crap and you know this then you are less likely to explore and find a better solution.
Quite frankly, we can't judge our own competence very well without the wisdom of age and careful observation & contemplation. Even then, we can only judge our competence relative to others.
This is assuming we have a working, widely accepted definition for competence which we tend to gloss over and use only in specific contexts.
I also strongly disagree with "it is a a wise man who knows he knows nothing". A wise man knows what he doesn't know, but he certainly knows some things. He knows where to apply his own judgment and where to apply the thinking of others.
One somewhat useful heuristic I've found for judging my own competence: If I think someone is wrong but I can't say why, I'm probably the incompetent one (they may be also). But if I think someone is wrong and I can give clear directions as to what they are doing wrong and how to fix it, I probably have a better understanding of the subject domain than they do.
It doesn't always work, and but it's especially handy when you turn it around and get someone else to tell you exactly why what you are doing is wrong.
This is a very, very dangerous heuristic. Competence does not imply the ability to articulate said competence. Many (most? all?) competences are in the doing, not the saying. Articulation is its own competence, and there's a bridge from one to the other, but only a bridge.
It is so dangerous because you are more likely to write someone off (yourself or others) if the articulation requirement isn't met. When it's others that you write off, you compound whatever latent Dunning-Kruger effect you may be experiencing. When it's yourself, you then open yourself to distortion caused by the Dunning-Kruger effect in others. eg, "he seems to know what he's talking about" etc.
"Most of us spend our lives steering ourselves toward the best of all possible futures, only to find that tomorrow rarely turns out as we had presumed. Why? As Harvard psychologist Daniel Gilbert explains, when people try to imagine what the future will hold, they make some basic and consistent mistakes. Just as memory plays tricks on us when we try to look backward in time, so does imagination play tricks when we try to look forward."
This can happen in the opposite extreme in the software engineering world. Only few best hackers see things through, and the remaining %99 argue with them missing the point that is very subtle.
The list of tags for this article is simply amazing:
ANOSOGNOSICS DILEMMA, BANK ROBBERIES, DECISION-MAKING, DONALD RUMSFELD, DUNNING-KRUGER EFFECT, INCOMPETENCE, KNOWLEDGE, LACK OF KNOWLEDGE
This actually had impressed me already at an early age (despite not really having read Socrates/Plato directly). I can only hope that it protected me against the DK effect at least a little bit.
Also note that Socrates ended up being poisoned by his enemies.
I also subscribe to the "all code sucks" variant of the theorem.
If barrels = the conventional tools that worked before, then that's not a bad way to start, so long as you stop after hitting your cranium on some brick walls and go and do something else. The answer often comes to me over breakfast the next day.
You can say that about smart people too. It is just hard to
jump outside the framework of your own education and experiences and come to a conclusion that conflicts with
everything you see as being right. He should have validated
his hypothesis by backing into his conclusion using several independent methods and not relying on just one.
The Dunning–Kruger effect is a cognitive bias in which "people reach erroneous conclusions and make unfortunate choices but their incompetence robs them of the metacognitive ability to realize it." The unskilled therefore suffer from illusory superiority, rating their own ability as above average, much higher than it actually is, while the highly skilled underrate their abilities, suffering from illusory inferiority. This leads to the perverse situation in which less competent people rate their own ability higher than more competent people. It also explains why actual competence may weaken self-confidence: because competent individuals falsely assume that others have an equivalent understanding. "Thus, the miscalibration of the incompetent stems from an error about the self, whereas the miscalibration of the highly competent stems from an error about others."
“In the modern world the stupid are cocksure while the intelligent are full of doubt.”
I have long believed, it's impossible to assess ones own intellect. What's more interesting is the ability of a genius to locate and acknowledge another genius even if he is from another field and even when it's not obvious to others.
The known unknowns are like a door you haven't opened. The unknown unknown is not even realizing there's a door. It's a thrill to discover such doors.
It makes me hopeful to think that just a couple of steps off the beaten track, there are miracles that the mainstream has not dreamed of (nor me). There are more things in heaven and earth, Horatio,/Than are dreamt of in your philosophy. Such is the motivation of a would-be scientist/inventor/entrepreneur.
One may laugh at such incompetent people as in the article... but how do we know we aren't incompetent? One way is objective tests: If we can pass a maths test, we likely know some maths. If our programs behaves as we predict, we likely know some coding. Another way is to consult with a community of people reputed to be competent, and ask them if we are. Universities do this - notwithstanding the study in the article - hopefully at least at the postgraduate level. I may be stupid, but at least I'm state-of-the-art-stupid. Of course, there are famous examples of radical scientists, scorned by the scientific establishment, who turned out to be right... (though perhaps famous more due to their appeal than their frequency.)