The Dunning-Kruger Effect Is Probably Not Real
McGill’s Office for Science and Society publishes short essays that separate sense from nonsense, and Jonathan Jarry started this one expecting a quick win: explain the effect, how it was discovered, what it means, end of story. Weeks of reading, a few statisticians, and a correspondence with David Dunning himself later, he concluded the celebrated effect is probably a data artefact. It’s the best short takedown I’ve read of a finding that survives on reputation alone.
The 1999 effect was specific and modest. Dunning and Kruger quizzed students on grammar, humour, and logic, comparing each guessed score to the real one. The worst performers overestimated themselves the most, and the best slightly underestimated. That gap, not “stupid people,” is what the paper actually claimed.
Dunning himself says the effect was never about “dumb people.” “The effect is about us, not them,” he wrote to Jarry. Pointing it at someone else inverts the original point: humility about your own blind spots.
The modern version is about being misinformed, not uninformed. Ask the boiling point of mercury and you know you don’t know. Ask the capital of Scotland and you say Glasgow with real confidence, and it’s Edinburgh. A confident wrong answer feels different from a blank, which made the effect easy to believe.
Random data reproduces the whole pattern. In 2016 and 2017 Ed Nuhfer’s team published two papers in Numeracy showing random computer-generated numbers produce the famous graph: bottom quartile overestimates, top quartile underestimates. Patrick McKnight, who once taught the effect, replicated the finding in R. A real bias of the brain should not appear when you remove the brain.
The artefact is measurement error plus binning. Self-assessment is noisy: your mood changes how you guess your score. Sorted into quartiles, that noise inflates the gap between the worst and best performers. Real effects shrink as measurement error grows; the Dunning-Kruger pattern gets sharper with more noise. “We have no instance in the history of scientific discovery where a finding improves by increasing measurement error. None.”
You might think this is another debunking that won’t hold up. Jarry pushed back at the critics for weeks, played devil’s advocate with two statisticians, and the result held. The related biases are real and separate: overconfidence, and the better-than-average effect where most drivers rate themselves above average. Losing Dunning-Kruger does not leave the brain spotless.
It persists because the criticism never traveled. Jarry found over 8,500 news hits endorsing the effect, from the New York Times to New Scientist, while academic critiques existed since the paper was published. The media learned the graph and the moral, and the incantation kept getting repeated.
The takeaway: the next time you reach for Dunning-Kruger to explain someone’s behaviour, ask whether you’ve read the effect or just the meme, then take the original lesson instead: be humble about the confidence gap in your own head. And before you cite any psychology finding in a meeting, ask whether it survives random data.
Related TMFNK Content
- Enhanced Rationality in Autism: What the Biases Literature Gets Wrong About Everyone Else The same biases literature that made Dunning-Kruger famous gets the same scrutiny: “predictably irrational” turns out to be a default, not a law.
- There Is No ‘Hard Problem of Consciousness’: Carlo Rovelli Dismantles the Mind-Body Gap Another celebrated idea that survived on reputation until someone checked the premise, Rovelli takes apart the hard problem the way McKnight takes apart the quartile graph.
- When Experience Fails to Yield Wisdom Overconfidence does not need Dunning-Kruger to exist; this one shows how accumulated experience breeds confidence that runs ahead of competence.
Crepi il lupo! 🐺