Skip to main content
Category: Effects
Type: Judgment bias
Origin: Stuart Oskamp, Journal of Consulting Psychology (1965); the three-way split in Don A. Moore and Paul J. Healy, Psychological Review (2008)
Also known as: Overconfidence bias
Quick Answer — The Overconfidence Effect is confidence that runs ahead of the record. Moore and Healy, in 2008, split that word into a score, a rank, and a range. A hard quiz pushed the score up and the rank down. An easy quiz reversed those two gaps, and the range stayed too narrow either way.

​
What is Overconfidence Effect?

The Overconfidence Effect is confidence in a judgment that exceeds what the record supports. The record might be a score, a rank against another person, or the share of times a stated range holds the truth.
On difficult tasks, people overestimate their actual performances but also mistakenly believe that they are worse than others; on easy tasks, people underestimate their actual performances but mistakenly believe they are better than others.
That sentence is from Don A. Moore and Paul J. Healy, Psychological Review, 2008. Overestimation is the score gap: the points you claim minus the points you earned. Overplacement is the rank gap. It starts from how far above another person you place yourself, then subtracts how far above that person you actually were. Overprecision is the range. Too much of the confidence sits on too few outcomes. The three names are parts of one literature. They are not three words for one miss. A practice test makes the split easy to see. It can ask what score you just earned, where that score sits against a random classmate, and how wide a band should still hold the true score. One warm feeling cannot fill three boxes. Bayesian Thinking is the formal habit of moving a prior toward new evidence. Moore and Healy use that habit as their account. After a noisy signal, the estimate should fall between the prior and the signal. The Overconfidence Effect, on this page, is what the three measures do when that move is imperfect, and when the band is tighter than its hits.

​
Overconfidence Effect in 3 Depths

  • Beginner: The cue is a sure feeling. Before you act on it, name which question it answered. The score, the rank, and the range are three questions.
  • Practitioner: Write the three answers on one line before a review. One is the score you think you earned. One is your rank against a random peer. One is a band you claim will hold the truth about nine times in ten. Later, score the band by its hits.
  • Advanced: A hard task and an easy task push the score and the rank in opposite directions. A prior that the new evidence only partly moves is not a prior that was dropped. Base Rate Neglect is the miss of leaving a stated share once a case arrives. The quiz below starts from a prior and pulls the estimate back toward it. A point on a 10-item quiz is not an office’s error rate.

​
Origin

The name is older than the three-way split. The split is what makes the name usable. In 1965, Stuart Oskamp published “Overconfidence in case-study judgments” in the Journal of Consulting Psychology, volume 29, pages 261–265. Oskamp was at Claremont Graduate School. The paper revises a talk given at the Western Psychological Association on 18 April 1964. Thirty-two judges read one published case, divided into four chronological stages. The case was Joseph Kidd, from Robert W. White, Lives in Progress (1952). After each stage, the judges answered the same 25 questions about the person. Eight judges were clinical psychologists at a California state hospital. Each had several years of clinical experience, and five had a doctorate. Eighteen were psychology graduate students. Six were advanced undergraduates in a personality class. None of them knew the hypotheses. They sat in groups of four to nine and worked at their own pace. The means that follow pool all 32 judges. Accuracy did not increase significantly as the stages added information. Confidence did. The paper reports confidence rising from 33 percent at stage 1 to 53 percent at stage 4. The average gap, confidence minus accuracy, was 7 points at stage 1 and 25 points at stage 4, t = 5.14, df = 31, far beyond the .001 level. All but 2 of the 32 judges became overconfident.
Clearly, increasing feelings of confidence are not a sure sign of increasing predictive accuracy about a case.
That is the paper’s closing line about this case file. The 33, the 53, the 7, and the 25 belong to these 25 questions. The hospital employed 8 of the judges. The figures are not a count of mistakes on its wards. Do not subtract 7 from 33, or 25 from 53, and call the result an accuracy percent. The confidence rise and the gaps are separate reports, and the rise is rounded in that sentence. A later line of studies asked for a range, often a 90 percent range, around a quantity. Moore and Healy summarize that line as ranges that hold the truth less than half the time. They cite Alpert and Raiffa (1982), Klayman, Soll, González-Vallejo, and Barlas (1999), and Soll and Klayman (2004). They also cite Teigen and Jørgensen (2005) for hit rates of 30 percent to 50 percent on 90 percent intervals. This page does not re-read those hit rates. They are not the quiz below. Moore and Healy put the three measures in one paper. “The Trouble With Overconfidence” is Psychological Review, volume 115, pages 502–517 (2008). Moore was at the Tepper School of Business, Carnegie Mellon University. Healy was in the Department of Economics at The Ohio State University. The quizzes were run in the laboratory at Carnegie Mellon. In a PsycINFO search reported there, roughly 64 percent of the empirical studies measured overestimation, roughly 5 percent measured overplacement, and roughly 31 percent measured overprecision. Roughly 74 percent of the overestimation studies asked how sure the person was about a single item. That format, the paper says, glues the score gap to the range.

​
Key Points

Three measures keep the Overconfidence Effect from collapsing into a mood. Statistical Thinking scores a range by the share of times it holds. The steps below are those three scores on the 2008 quizzes.
1

Name which question the feeling answers

A sure feeling is not yet a score, a rank, or a range. Overestimation is the score you claim minus the score you earned. Overplacement starts from how far above the other person you place yourself, then subtracts how far above that person you actually were. Overprecision is a range that catches the truth less often than the confidence attached to it.
2

Read a hard task as two opposite gaps

On the hard quizzes, the mean score was 0.71 out of 10, and the mean estimate of one’s own score was 1.50. Overestimation averaged 0.79 points, with a standard deviation of 1.50. Overplacement averaged minus 1.36 points, with a standard deviation of 2.39. The score was too high. The rank, once the actual gap was removed, was too low.
3

Read an easy task as the reverse pair

On the easy quizzes, the mean score was 8.86 out of 10, and the mean estimate was 8.64. Overestimation averaged minus 0.22 points, with a standard deviation of 0.93. Overplacement averaged 0.48 points, with a standard deviation of 2.59. Medium quizzes sat between them: a score gap of 0.01 and a rank gap of 0.04, which the paper treats as neither over nor under.
4

Score the range by its hits

The range did not flip with difficulty. From each distribution over the scores 0 through 10, the band closest to a 90 percent interval covered 3.11 scores on average and stood for 90.5 percent confidence. It sits above 90 because 20 percent of the reports put all the weight on one score. The true score landed inside 73.1 percent of the time: 55.1 percent before the quiz, 85.6 percent after it, and 84.4 percent after grading, for the other person’s score.

​
Applications

Use the three lines whenever a judgment will be treated as a fact. Keep the quiz numbers as the picture of the pattern. Leave them in the lab when you leave the lab.

Write three lines before the review

For a course paper, a forecast, or a promotion note, write the score you are claiming, the rank against a random peer, and a band you want to hold nine times in ten. In the 2008 lab, the prior was not an inflated rank. The expected score was 5.36 out of 10 for the self and 5.36 for a random earlier participant. The gap showed up after the quiz, and the score and the rank moved in opposite directions.

Sort the task before you sort the person

A hard exam, a rare call, or a long shot follows the hard pattern: the claimed score runs high, and the claimed rank runs low. A familiar task follows the easy pattern. There the score was 0.22 points too low and the rank was 0.48 points too high. Those are trivia points on a 10-item quiz. They are not a grade curve, and they are not a rate of bad hires.

Draw the band before the file grows

Oskamp’s judges moved from 33 percent confidence to 53 percent, while accuracy did not rise significantly. In 2008, the band’s hit rate was 55.1 percent before the quiz and 85.6 percent after it. Write the band before the extra pages arrive, then score the band. Added material is a reason to recompute the three lines. It is not, by itself, a reason to tighten the band.

Keep a two-choice hunch off the range

Fischhoff, Slovic, and Lichtenstein, in 1977, ran five studies with 528 paid volunteers. People picked one of two answers, then stated a probability from 50 percent to 100 percent. One item asked whether absinthe is a liqueur or a precious stone. Moore and Healy say that format glues a too-high score to a too-narrow range. If the decision needs a range, ask for the band. This page does not print how often a 100 percent answer was wrong.

​
Case Study

The counted quiz is the experiment in Moore and Healy’s 2008 paper. Eighty-two Carnegie Mellon students took part in the laboratory. Their mean age was 24 years, the standard deviation was 6.42, and the range was 18 to 59. The posting that recruited them was titled “Tons o’ Trivia.” The authors worried that a trivia title might attract people who would place themselves too high. That worry does not show up in the prior. The mean expected score was 5.36 for the self and 5.36 for a random earlier participant. Each person took 18 quizzes of 10 items. The topics were science, movies, history, sports, geography, and music. Each topic had an easy quiz, a medium quiz, and a hard quiz. Order was random, with one of each difficulty inside every block of three, across six blocks. Mean scores were 8.86, 5.92, and 0.71, F(2, 162) = 781.00, p < .001. People saw the difference. Interim estimates of their own scores were 8.64, 5.93, and 1.50, F(2, 162) = 586.00, p < .001. The shift from easy to hard was smaller in the beliefs than in the scores. For every score from 0 to 10, they set a probability, for themselves and for a random earlier participant. The eleven bars always summed to 100 percent. A quadratic scoring rule paid the reports. The paper says the rule pays the most when the report matches the person’s honest best estimate, from 0to0 to 2 for each set of guesses. Pay for the quiz itself depended on percentile rank. Five of the 18 rounds were chosen at random and averaged. Mean pay was $20.23 after about 90 minutes. The score gap and the rank gap moved apart. At the interim, after the quiz and before the answers, overestimation averaged minus 0.22 points on easy quizzes, 0.01 on medium quizzes, and 0.79 on hard quizzes, F(2, 162) = 42.53, p < .001. The standard deviations on those three means were 0.93, 1.27, and 1.50. Overplacement averaged 0.48, 0.04, and minus 1.36, F(2, 162) = 24.88, p < .001. Those standard deviations were 2.59, 3.91, and 2.39. Inside a single quiz the two measures moved together, with an average correlation of .29. Across the 18 quizzes they moved apart, r(18) = −.64, p = .004. Rounds of practice did not erase the score gap. The paper reports no main effect of block. A block-by-difficulty interaction was significant, F(10, 810) = 2.65, p = .003, and the paper says the change was not a straight line. Tests for a linear pattern were not significant. Overplacement showed no main effect of block and no block-by-difficulty interaction. The range stayed too tight on the paper’s own test. The bands that stood for 90.5 percent confidence held the true score 73.1 percent of the time. The paper’s binomial test puts the odds of a hit rate that low, for bands of that confidence, at about 6 per million. Confidence ran ahead of the hit rate for every difficulty, every round, every phase, and both the self and the other person, all ps < .01. Wider bands belonged to less accurate estimates. At the interim, band width correlated with distance from the true score, r = .54 for the self and r = .23 for the other person, both p < .001. The lesson is the split. One word had been asked to cover a score, a rank, and a range. On these quizzes the first two ran in opposite directions, and the range did not follow either one. The students were answering trivia and were paid to report a distribution. A mean of 0.79 points on a 10-item quiz is not 79 percent. The hit rate of 73.1 percent is not the 30-to-50 percent summary of the older interval studies. The $20.23 is average pay for the session. It is not a return in a market.

​
Boundaries and Failure Modes

The Overconfidence Effect is the wrong name when the three measures are folded into one mood, and when a quiz point is carried out of the lab as a rate at work. The first boundary is the hard-easy reversal. Easy quizzes produced a score that was 0.22 points too low and a rank that was 0.48 points too high. Hard quizzes produced a score that was 0.79 points too high and a rank that was 1.36 points too low. The standard deviations are larger than those means. The shift is in the average. Dunning-Kruger Effect is about skill and the ability to judge that skill. The hard-quiz mean of 0.71 is what these items produced. Moore and Healy cite Kruger and Dunning (1999) as one report of underestimation among people who are skilled. This page does not re-read that 1999 paper, and it does not move the 0.79 or the minus 0.22 onto it. The second boundary is the range. Seeing the quiz raised the hit rate from 55.1 percent to 85.6 percent. After grading, the hit rate for the other person’s score was 84.4 percent. All three sit under the 90.5 percent confidence of the bands. Moore and Healy’s summary of older 90 percent intervals, a hit rate under half the time, and the 30-to-50 percent range they attribute to Teigen and Jørgensen (2005), are a different method. The 73.1 percent is a band cut from a full distribution on a score from 0 to 10. Oskamp’s rise from 33 percent confidence to 53 percent, on a flat accuracy line, is a third task. Added pages raised confidence there. Those pages are not the 2008 hit rates. The third boundary is export, including the neighbors. Planning Fallacy is about time and cost on one’s own project. Optimism Bias is about one’s own future relative to other people. Moore and Healy discuss both as places where a high or low chance can flip estimation and placement. This page does not carry their numbers. Illusory Superiority is the page for placing oneself above others on an easy trait. Illusion of Control is the page for how much a person thinks they can steer an outcome. Illusion of Validity is the page for staying sure of cues that predict poorly. The California state hospital employed 8 of Oskamp’s judges. The 7-point gap and the 25-point gap are not a ward’s error rate.

​
Common Misconceptions

Three confusions travel with the word. The three measures get treated as one feeling. More pages get treated as more accuracy. A quiz point gets treated as a rate at work.
Across the 18 quizzes the correlation was r(18) = −.64, p = .004. Inside one quiz the average correlation was .29, because both measures use the person’s own expected score. The finding is the opposite sign across quizzes. Easy items lowered the score gap and raised the rank gap. Hard items did the reverse. A single feeling of being sure does not describe those two means.
Oskamp’s accuracy did not increase significantly from stage 1 to stage 4. Confidence went from 33 percent to 53 percent. The gap, confidence minus accuracy, went from 7 points to 25 points, t = 5.14, df = 31. All but 2 of the 32 judges became overconfident. Extra pages are a reason to recompute the score, the rank, and the range. They are not a reason to trust the feeling that grew with the file.
The 0.79, the minus 0.22, the 0.48, and the minus 1.36 are mean gaps on a 10-item trivia quiz taken by 82 students. The 73.1 percent is the hit rate of a band on that quiz, against bands that stood for 90.5 percent confidence. The 33 percent and the 53 percent are confidence on 25 questions about one published case. None of these figures is a count of bad trades, missed diagnoses, or bad hires.
The Overconfidence Effect sits next to a skill pattern, a time pattern, a belief about one’s future, and the formal way of moving a prior. The links are neighbors. They are not other names for this gap.

Dunning-Kruger Effect

A pattern in which low skill and self-assessment fail together. Moore and Healy cite the 1999 paper as one report of underestimation among skilled people. The quiz on this page separates score, rank, and range by item difficulty.

Planning Fallacy

Underestimating the time or cost of one’s own project. A short task can produce too high a time estimate, which is a low estimate of performance. That case stays on its own page.

Optimism Bias

A belief that one’s own future compares well with other people’s futures. Placement against a peer is one piece of the 2008 rank measure. The future-self case is a different page.

Base Rate Neglect

Underusing a stated share once a case arrives. Moore and Healy’s account pulls the estimate back toward the prior. Using a prior in a regressive way is a different miss from leaving it.

Bayesian Thinking

Moves a prior toward new evidence. The 2008 theory says a noisy signal should leave the estimate between the prior and the signal. The prior mean on the quiz was 5.36 for both people.

Self-Serving Bias

Taking credit for success and handing failure to the situation. That is a story about causes. The three measures here are a score, a rank, and a range, scored against the record.

​
One-Line Takeaway

Write the score, the rank, and the range as three answers, and score the range by how often it holds.