> And D-K did indeed show a correlation between the two, just not as strong of one as we would expect. Rather, they showed a consistent bias. That's the interesting result.
Right, so:
1. If the data were truly random, with no correlation, we'd expect the line to be straight across the middle, with the first quartile at 50% and the last quartile also at 50%
2. If the data were 100% accurate and precise [1], we'd expect the line to be diagonal, with the first quartile at 12.5% and the last quartile at 87.5%.
3. If the data were accurate but not precise (i.e., basically right but with some randomness built in), we'd expect the line to be in between #1 and #2 -- basically, changing from #2 into #1 as the randomness increases, but with the intersection at 50%.
That's because someone in the 2nd percentile can't underestimate themselves as much as they can overestimate themselves, and someone in the 98th percentile can't oversetimate themselves as much as they can underestimate themselves. But in any case, the "0 bias" case looks symmetric.
4. But what we actually see is none of the above: we see the 1st quartile being at (eyeballing the chart) 60%, and the last quartile at 75%.
That shows that there is indeed some ability for self-evaluation, but it's off. The fourth quartile could indeed just be random, the effect of clipping at the top meaning that the upper quartile cannot overestimate themselves as much as they understimate themselves. But there's no getting around the fact that the bottom quartile are overestimating themselves.
> But there's no getting around the fact that the bottom quartile are overestimating themselves.
It's because higher competence goes along with more accurate self-assessment but not less bias. So the high performers underestimate with less magnitude than the low performers overestimate, but they both under and over estimate themselves with the same frequency.
Right, so:
1. If the data were truly random, with no correlation, we'd expect the line to be straight across the middle, with the first quartile at 50% and the last quartile also at 50%
2. If the data were 100% accurate and precise [1], we'd expect the line to be diagonal, with the first quartile at 12.5% and the last quartile at 87.5%.
3. If the data were accurate but not precise (i.e., basically right but with some randomness built in), we'd expect the line to be in between #1 and #2 -- basically, changing from #2 into #1 as the randomness increases, but with the intersection at 50%.
That's because someone in the 2nd percentile can't underestimate themselves as much as they can overestimate themselves, and someone in the 98th percentile can't oversetimate themselves as much as they can underestimate themselves. But in any case, the "0 bias" case looks symmetric.
4. But what we actually see is none of the above: we see the 1st quartile being at (eyeballing the chart) 60%, and the last quartile at 75%.
That shows that there is indeed some ability for self-evaluation, but it's off. The fourth quartile could indeed just be random, the effect of clipping at the top meaning that the upper quartile cannot overestimate themselves as much as they understimate themselves. But there's no getting around the fact that the bottom quartile are overestimating themselves.
[1] https://en.wikipedia.org/wiki/Accuracy_and_precision