Outcome Bias

​

What is Outcome Bias?

Outcome Bias is a shift in the judged quality of a decision that follows the result, after the chances were already on the page. The thinking, the options, and the stated odds stay the same. The ending changes the score.

An outcome bias existed when the evaluation of the decisions depended on their outcomes.

That sentence is how Baron and Hershey close their definition in the Journal of Personality and Social Psychology, 1988, pages 569–579. They looked for people taking a result into account in a way that is irrelevant to the quality of the decision. The judge was told the probabilities the decision maker had. The result then added no new fact about those probabilities. A fair coin called before the toss is the everyday picture. Heads and tails were each one in two. The call does not become sharper thinking because the coin landed heads. Hindsight Bias is the neighbor that moves the remembered chance. This page keeps the chance fixed and moves the grade given to the thinking.

​

Outcome Bias in 3 Depths

• Beginner: Before you praise or blame a choice, write the chance that was known at the time. A smooth recovery and a rare harm can share that same chance.
• Practitioner: Score the information on one line and the result on a second line. Finish the first line before you read the ending aloud in a review, a family talk, or a class debrief.
• Advanced: A result grades the choice when it reveals something the chooser did not have. One known chance does not do that. A run of similar unknown chances can. A loss can also be the better inference, once the missed option is allowed to teach.

​

Origin

The habit is old. The name, and the test that holds the odds still, are from 1988. Baron and Hershey open with Antoine Arnauld. The Dickoff and James translation of The Art of Thinking, the Port-Royal Logic, gives the 1662 remark on page 285. Arnauld calls it a fault, condemned but seldom avoided, to evaluate the intention of an act by the act’s outcome. People fail to separate fortune from wisdom, and misfortune from guilt. That sentence is the ancestor. It is not a counted study, and it is not the modern name. The decision-analysis norm they quote is Ward Edwards, 1984, page 7. Edwards calls it a familiar elementary point.

A good decision cannot guarantee a good outcome.

His next lines say that a real decision is a bet, so the grade depends on the stakes and the odds. Baron and Hershey ask how well people outside that profession keep the distinction. The paper that names the bias is “Outcome bias in decision evaluation,” volume 54, number 4, pages 569–579, the April 1988 issue. Baron was in the Psychology Department at the University of Pennsylvania. Hershey was in the Department of Decision Sciences. Both were senior fellows of the Leonard Davis Institute for Health Economics. The work was supported by the National Institute of Mental Health, grant MH37241, and by the National Science Foundation, grant SES-8509807. They thank Mark Spranca. They sort information into three piles. Actor information is known to the decision maker and not to the judge. Judge information is known only to the judge. Joint information is known to both. In their cases the result is judge information. The stated odds are joint information. A result can still be useful when it hints at facts the chooser had and the judge lacks. A result can also teach a rate across many similar decisions, as draws from a deck of unknown composition can. Their test removes those uses. The judge is told everything relevant that the chooser had. If the score still follows the ending, that leftover is the Outcome Bias. They ran five studies with undergraduates. The decisions were medical cases or money gambles. People rated the quality of thinking, the competence of the decision maker, or their willingness to let that person decide for them. They understood that they already had the relevant information. Favorable results received the better ratings. In the money gambles, a poor result for the option not chosen also raised the rating of the choice that was made. People who were asked said they should not use the result. They used it anyway. The authors’ partial account is salience. A death on the table makes the risk of death easier to see as an argument against surgery. That account did not cover every set of items. Baruch Fischhoff’s 1975 hindsight studies moved judged probabilities. Baron and Hershey held the probabilities still, so a change in the rated thinking would not have to travel through a changed chance. The two effects can stack in ordinary life. The pages stay separate. Just-World Hypothesis is the further neighbor that wants a bad ending to have been deserved.

​

Key Points

Four checks keep the Outcome Bias tied to a result that was not needed. Each number stays inside the study that printed it.

​

Applications

Use the stated chance, the second line, and the option not taken. Leave each printed gap inside its own study.

​

Case Study

The counted picture is a bypass operation, told twice, thirty-five years apart. The decision and the stated risk stay fixed. The ending changes. In Experiment 1, Baron and Hershey recruited twenty undergraduates at the University of Pennsylvania through a sign on a campus walkway. They were paid by the hour. Ten read the cases in one order, and ten read them in reverse. Each person saw both endings, with the matched cases placed far apart in a list of fifteen. Many said they remembered an earlier answer and repeated it. The authors note that this within-subjects design makes a small effect easier to detect and also shrinks it. The heart case concerned a 55-year-old man who had stopped working because of chest pain. He enjoyed his work and did not want to stop. Pain also interfered with travel and recreation. A bypass would relieve the pain and increase life expectancy from age 65 to age 70. Eight percent of people who have the operation die from the operation itself. A footnote says the 8 percent was chosen from pilot data so the decision would look difficult. In one version the physician decided. In the matched version the man decided. Success meant the operation succeeded. Failure meant it failed and the man died. On the scale from 3 to −3, the physician’s decision averaged 0.85 after success, with a standard deviation of 1.62, and −0.05 after failure, with a standard deviation of 1.77. The patient’s decision averaged 1.00 after success and 0.75 after failure, with standard deviations of 1.05 and 1.26. Across the seven success-failure pairs, including liver surgery and two testing problems, the mean advantage for success was 0.70. The patient-only pairs, heart and liver together, still showed a mean of 0.48, t = 2.59, df = 19, p < .01. In 17 of those 40 pairs the success case scored higher, and in 4 the failure case did. The same vignette was repeated in 2023 by Sriraj Aiyer, Hoi Ching Kam, Ka Yuk Ng, Nathaniel A. Young, Jiaxin Shi, and Gilad Feldman, in the International Review of Social Psychology, volume 36, issue 1, article 12. They preregistered a between-subjects version of this one case, not the full set of fifteen. Each person saw one combination of decision maker and ending. Recruits came from Amazon Mechanical Turk through CloudResearch. Of 707 people, 705 finished. After the other preregistered exclusions, 692 remained, 364 of them women, with a mean age of 38.9 and a range from 19 to 78. They had to pass four comprehension checks before the rating, including the question of whether 8 percent would die. The means on the same style of scale were 1.81 for a physician and success, 1.76 for a patient and success, 0.45 for a physician and failure, and 0.90 for a patient and failure. The cell sizes were 173, 171, 172, and 176. The success-versus-failure gap was 1.36 for the physician, Cohen’s d = 1.10, and 0.86 for the patient, d = 0.77. The 95 percent intervals run from 0.94 to 1.26 and from 0.62 to 0.93. Both Welch tests were one-tailed, and both printed p < .001. The authors call the shift from within-subjects to between-subjects a close-to-far replication. They estimate the 1988 paired effect as d = 0.21 for the patient, with a 95 percent interval from −0.23 to 0.66. They estimate d = 0.53 for the physician, with an interval from 0.06 to 0.99. Those d values are their calculation. The 1988 paper prints the t tests. A paired d and an independent d are not the same comparison, which the 2023 paper says in its note under the table. The lesson is local. With the 8 percent already printed, a success still received the higher rating of the thinking, in a small paid student sample and again in a larger online sample that saw only one ending. The standard deviations in 1988 are larger than the case means. The 2023 intervals on the old paired d values are wide. Neither study is a record of operations performed in a hospital.

​

Boundaries and Failure Modes

Outcome Bias is the wrong summary in three places. A result can reveal a hidden fact. The question can be punishment rather than the quality of thinking. A lab gap can also be carried into a hospital or a firm. The first boundary is information. Baron and Hershey say a judge who lacks the chooser’s probabilities may treat the result as a clue to those probabilities. In the extreme, with no other information, a bad result is a reasonable first hypothesis of a badly made decision. They also say a sequence of similar decisions can teach a rate. Their bias claim starts after those uses are removed. Experiment 2 used 41 paid students and a scale from −30 to 30. The gaps were 4.97 for physician surgery, t = 3.15, df = 38, and 4.15 for patient surgery, t = 2.21, df = 40. The testing cases differed by 3.50, t = 3.80, df = 40. The salience account, that a good result makes the supporting reasons look more important, was reliable only for the physician surgery items, t = 3.61, df = 38. It was not reliable for the patient items or the testing items. The authors write that they cannot explain that split, and that salience cannot fully explain the bias. The second boundary is the question being asked. Elaine Walster’s 1966 study, which they cite, had people judge a driver as more responsible when the damage was more severe. Responsibility and punishment can track harm even when the thinking is held still, because the judge often does not know what the driver knew. Their own question was the quality of thinking, with the odds supplied. Experiment 4 then asked a second question, future competence, and excluded people who said the result should count. The competence gap remained, and it correlated 0.85, across subjects, with the decision-quality gap. An ambiguous risk, one study at 12 percent and another at 4 percent, lowered competence ratings regardless of the ending, t = 2.46, df = 24, two-tailed. It did not reliably change the size of the outcome gap. The studies rate someone else’s decision. The discussion says they did not test a person grading their own choice. Fundamental Attribution Error is the page for reading a person from a single act. Illusion of Control is the page for treating a chance result as a personal skill. Some of the 1988 comments treat luck as a stable trait, and the footnote says those examples include a pilot questionnaire. The third boundary is export. The mean of 0.70, the heart-case means, the 9.91, the 5.03, the 3.41, and the 2023 values of d are ratings of written cases. They are not surgical error rates, malpractice findings, or promotion gaps. Experiment 5’s willingness counts come from 111 students present in a class Baron taught. Two baseline sheets that said no and then yes were dropped, while Table 5’s four rows also sum to 111. Those two facts stay side by side. The mystery-first lose row is 3 increases, 12 decreases, and 14 unchanged. Of the 23 competence remarks that were not “average,” wins drew 6 above average and 4 below, and losses drew 2 above and 11 below. That is a class session, not a hiring study. The 2023 analysis of people who rejected the result used 44 of the 705 people who finished the survey. The later sample of 692 is a separate cut. The 95 percent interval from 0.21 to 1.08 belongs to the mean difference of 0.75. The abstract prints that interval beside d. The results paragraph prints it beside the difference.

​

Common Misconceptions

Three shortcuts travel with the name. A stated principle is treated as a cure. A moved chance is treated as the whole effect. A bad ending is treated as proof that the thinking was bad.

​

Related Concepts

Outcome Bias sits next to a rewritten chance, a person read from one act, and a chance result read as skill. It also sits next to a record that keeps only visible endings, a demand that harm be deserved, and a search for reasons after the ending is known.

​

One-Line Takeaway