Probabilistic Coherence Weighting for Optimizing Expert Forecasts

References

• Brier GW (1950) Verification of forecasts expressed in terms of probability. Monthly Weather Rev. 78(1):1–3.Crossref, Google Scholar
• Bruine de Bruin W, Fischbeck PS, Stiber NA, Fischhoff B (2002) What number is “fifty–fifty”?: Redistributing excessive 50% responses in elicited probabilities. Risk Anal. 22(4):713–723.Crossref, Google Scholar
• Burgman MA, McBride M, Ashton R, Speirs-Bridge A, Flander L, Wintle B, Fidler F, Rumpff L, Twardy C (2011) Expert status and performance. PLoS One 6(7):1–7.Crossref, Google Scholar
• Clemen RT (2008) Comment on Cooke's classical method. Reliability Engrg. System Safety 93:760–765.Crossref, Google Scholar
• Clemen RT, Winkler RL (1999) Combining probability distributions from experts in risk analysis. Risk Anal. 19(2):187–203.Crossref, Google Scholar
• Cooke RM (1991) Experts in Uncertainty (Oxford University Press, Oxford, UK).Google Scholar
• Cooke RM, Goossens LLHJ (2008) TU Delft expert judgment data base. Reliability Engrg. System Safety 93(5):657–674.Crossref, Google Scholar
• De Finetti B (1990) Theory of Probability: A Critical Introductory Treatment (John Wiley & Sons, New York).Google Scholar
• Genest C, McConway KJ (1990) Allocating the weights in the linear opinion pool. J. Forecasting 9(1):53–73.Crossref, Google Scholar
• Gneiting T (2011) Making and evaluating point forecasts. J. Amer. Statist. Assoc. 106(494):746–762.Crossref, Google Scholar
• Herzog SM, Hertwig R (2009) The wisdom of many in one mind: Improving individual judgments with dialectical bootstrapping. Psych. Sci. 20(2):231–237.Crossref, Google Scholar
• Hirt ER, Markman KD (1995) Multiple explanation: A consider-an-alternative strategy for debiasing judgments. J. Personality Soc. Psych. 69(6):1069–1086.Crossref, Google Scholar
• Hsee C (1996) The evaluability hypothesis: An explanation for preference reversals between joint and separate evaluations of alternatives. Organ. Behav. Human Decision Processes 67(1):247–257.Crossref, Google Scholar
• Karvetski CW, Olson KC, Gantz DT, Cross GA (2013) Structuring and analyzing competing hypotheses with Bayesian networks for intelligence analysis. EURO J. Decision Processes, ePub ahead of print April 30, http://link.springer.com/article/10.1007/s40070-013-0001-x.Crossref, Google Scholar
• Kolmogorov A (1956) Foundations of the Theory of Probability (Chelsea Publishing Company, New York).Google Scholar
• Larrick RP, Soll JB (2006) Intuitions about combining opinions: Misappreciation of the averaging principle. Management Sci. 52(1):111–127.Link, Google Scholar
• Lindley DV, Tversky A, Brown RV (1979) On the reconciliation of probability assessments. J. Roy. Statist. Soc. 142(2):146–180.Crossref, Google Scholar
• Lord CG, Lepper MR, Preston E (1984) Considering the opposite: A corrective strategy for social judgment. J. Personality Soc. Psych. 47(6):1231–1243.Crossref, Google Scholar
• Macchi L, Osherson D, Krantz DH (1999) A note on superadditive probability judgment. Psych. Rev. 106(1):210–214.Crossref, Google Scholar
• Mandel DR (2005) Are risk assessments of a terrorist attack coherent? J. Experiment. Psych.: Appl. 11(4):277–288.Crossref, Google Scholar
• Mandel DR (2008) Violations of coherence in subjective probability: A representational and assessment process account. Cognition 106(1):130–156.Crossref, Google Scholar
• Merrick JRW (2008) Getting the right mix of experts. Decision Anal. 5(1):43–52.Link, Google Scholar
• Murphy AH (1973) A new vector partition of the probability score. J. Appl. Meteorology 12(4):595–600.Crossref, Google Scholar
• Oppenheimer DM, Meyvis T, Davidenko N (2009) Instructional manipulation checks: Detecting satisficing to increase statistical power. J. Experiment. Soc. Psych. 45(4):867–872.Crossref, Google Scholar
• Osherson D, Vardi MY (2006) Aggregating disparate estimates of chance. Games Econom. Behav. 56(1):148–173.Crossref, Google Scholar
• Pate-Cornell ME (1996) Uncertainties in risk analysis: Six levels of treatment. Reliability Engrg. Systems Safety 54(2–3):95–111.Crossref, Google Scholar
• Peters E, Dieckmann NF, Dixon A, Hibbard JH, Mertz CK (2007) Less is more in presenting quality information to consumers. Medical Care Res. Rev. 64(2):169–190.Crossref, Google Scholar
• Predd JB, Osherson DN, Kulkarni SR, Poor HV (2008) Aggregating probabilistic forecasts from incoherent and abstaining experts. Decision Anal. 5(4):177–189.Link, Google Scholar
• Rohrer D, Pashler H (2007) Increasing retention without increasing study time. Current Directions Psych. Sci. 16(4):183–186.Crossref, Google Scholar
• Sharp GL, Cutler BL, Penrod SD (1988) Performance feedback improves the resolution of confidence judgments. Organ. Behav. Human Decision Processes 42(3):271–283.Crossref, Google Scholar
• Sieck WR, Merkle EC, Van Zandt T (2007) Option fixation: A cognitive contributor to overconfidence. Organ. Behav. Human Decision Processes 103(1):68–83.Crossref, Google Scholar
• Sprenger AM, Dougherty MR, Atkins SM, Franco-Watkins AM, Thomas RP, Lange N, Abbs B (2011) Implications of cognitive load for hypothesis generation and probability judgment. Frontiers Psych. 2(129):1–15.Google Scholar
• Surowiecki J (2005) The Wisdom of Crowds (Double Day, New York).Google Scholar
• Tetlock PE (2005) Expert Political Judgment: How Good Is It? How Can We Know? (Princeton University Press, Princeton, NJ).Google Scholar
• Tsai J, Kirlik A (2012) Coherence and correspondence competence: Implications for elicitation and aggregation of probabilistic forecasts of world events. Proc. Human Factors and Ergonomics Soc. 56th Annual Meeting (Sage, Thousand Oaks, CA), 313–317.Crossref, Google Scholar
• Tversky A, Kahneman D (1974) Judgment under uncertainty: Heuristics and biases. Science 185(4157):1124–1131.Crossref, Google Scholar
• Tversky A, Koehler DJ (1994) Support theory: A nonextensional representation of subjective probability. Psych. Rev. 101(4):547–567.Crossref, Google Scholar
• Vul E, Pashler H (2008) Measuring the crowd within: Probabilistic representations within individuals. Psych. Sci. 19(7):645–647.Crossref, Google Scholar
• Wang G, Kulkarni SR, Poor HV, Osherson DN (2011) Aggregating large sets of probabilistic forecasts by weighted coherent adjustment. Decision Anal. 8(2):128–144.Link, Google Scholar
• Weller JA, Dieckmann NF, Tusler M, Mertz CK, Burns WJ, Peters E (2012) Development and testing of an abbreviated numeracy scale: A Rasch analysis approach. J. Behavioral Decision Making 26(2):198–212.Crossref, Google Scholar
• Williams JJ, Mandel DR (2007) Do evaluation frames improve the quality of conditional probability judgment? McNamara DS, Trafton JG, eds. Proc. 29th Annual Meeting of the Cognitive Sci. Soc. (Lawrence Erlbaum Associates, Inc., Mahwah, NJ), 1653–1658.Google Scholar
• Winkler RL, Clemen RT (2004) Multiple experts vs. multiple methods: Combining correlation assessments. Decision Anal. 1(3):167–176.Link, Google Scholar
• Wright G, Rowe G, Bolger F, Gammack J (1994) Coherence, calibration, and expertise in judgmental probability forecasting. Organ. Behav. Human Decision Processes 57(1):1–25.Crossref, Google Scholar
• Yaniv I, Yates JF, Smith JEK (1991) Measures of discrimination skill in probabilistic judgment. Psych. Bull. 110(3):611–617.Crossref, Google Scholar