Methods for Eliciting Informative Prior Distributions: A Critical Review

References

• Abbas AE, Budescu DV, Yu HT, Haggerty R (2008) A comparison of two probability encoding methods: Fixed probability vs. fixed variable values. Decision Anal. 5(4):190–202.Link, Google Scholar
• Akbarov A (2009) Probability elicitation: Predictive approach. PhD thesis, University of Salford, Salford, United Kingdom.Google Scholar
• Albert I, Donnet S, Guihenneuc-Jouyaux C, Low-Choy S, Mengersen K, Rousseau J (2012) Combining expert opinions in prior elicitation. Bayesian Anal. 7(3):503–532.Crossref, Google Scholar
• Arias-Nicolás JP, Ruggeri F, Suárez-Llorens A (2016) New classes of priors based on stochastic orders and distortion functions. Bayesian Anal. 11(4):1107–1136.Crossref, Google Scholar
• Aspinall W (2010) A route to more tractable expert advice. Nature 463(7279):294–295.Crossref, Google Scholar
• Baron RS (2005) So right it’s wrong: Groupthink and the ubiquitous nature of polarized group decision making. Adv. Experiment. Soc. Psych. 37(2):219–253.Crossref, Google Scholar
• Barons MJ, Mascaro S, Hanea AM (2022) Balancing the elicitation burden and the richness of expert input when quantifying discrete bayesian networks. Risk Anal., ePub ahead of print June 19, https://doi.org/10.1111/risa.13772.Google Scholar
• Basu S (1994) Variations of posterior expectations for symmetric unimodal priors in a distribution band. Sankhya¯: Indian J. Statist, Ser. A (1961–2002), 56(2):320–334.Google Scholar
• Brockhoff K (1975) IV. E. The Performance of Forecasting Groups in Computer Dialogue and Face-to-face Discussion, vol. 68 (Addison Wesley Publishing Company).Google Scholar
• Cagno E, Caron F, Mancini M, Ruggeri F (2000) Using AHP in determining the prior distributions on gas pipeline failures in a robust Bayesian approach. Reliable Engrg. Systems Safety 67(3):275–284.Crossref, Google Scholar
• Casement CJ, Kahle DJ (2018) Graphical prior elicitation in univariate models. Comm. Statist. Simulation Comput. 47(10):2906–2924.Crossref, Google Scholar
• Chen MH, Manatunga AK, Williams CJ (1998) Heritability estimates from human twin data by incorporating historical prior information. Biometrics 54(4):1348–1362.Crossref, Google Scholar
• Chesley G (1975) Elicitation of subjective probabilities: A review. Accounting Rev. 50(2):325–337.Google Scholar
• Clemen RT, Winkler RL (1999) Combining probability distributions from experts in risk analysis. Risk Anal. 19(2):187–203.Crossref, Google Scholar
• Cooke RM, et al. (1991) Experts in Uncertainty: Opinion and Subjective Probability in Science (Oxford University Press).Crossref, Google Scholar
• Diamond IR, Grant RC, Feldman BM, Tomlinson GA, Pencharz PB, Ling SC, Moore AM, et al. (2014) Expert beliefs regarding novel lipid-based approaches to pediatric intestinal failure–associated liver disease. J. Parenteral Enteral Nutrition 38(6):702–710.Crossref, Google Scholar
• Eckenrode RT (1965) Weighting multiple criteria. Management Sci. 12(3):180–192.Link, Google Scholar
• Edwards W, Barron FH (1994) Smarts and smarter: Improved simple methods for multiattribute utility measurement. Organ. Behav. Human Decision Processes 60(3):306–325.Crossref, Google Scholar
• Exley CL, Kessler JB (2019) The gender gap in self-promotion. Technical report, National Bureau of Economic Research, Cambridge, MA.Google Scholar
• Ferrell WR (1994) Discrete subjective probabilities and decision analysis: Elicitation, calibration and combination. Wright G, Ayton P, eds. Subjective Probability (John Wiley & Sons), 411–451.Google Scholar
• Galway LA (2007) Subjective Probability Distribution Elicitation in Cost Risk Analysis: A Review (RAND Corporation, Santa Monica), TR-410-AF.Google Scholar
• Garthwaite PH, Kadane JB, O’Hagan A (2005) Statistical methods for eliciting probability distributions. J. Amer. Statist. Assoc. 100(470):680–701.Crossref, Google Scholar
• Genest C, Zidek JV (1986) Combining probability distributions: A critique and an annotated bibliography. Statist. Sci. 1(1):114–135.Crossref, Google Scholar
• Gore SM (1987) Biostatistics and the medical research council. Medical Res. Council News 35:19–20.Google Scholar
• Hanea AM, Hemming V, Nane GF (2022) Uncertainty quantification with experts: Present status and research needs. Risk Anal. 42:254–263.Crossref, Google Scholar
• Hanea A, McBride M, Burgman M, Wintle B (2018) Classical meets modern in the idea protocol for structured expert judgement. J. Risk Res. 21(4):417–433.Crossref, Google Scholar
• Hartmann M, Agiashvili G, Bürkner P, Klami A (2020) Flexible prior elicitation via the prior predictive distribution. Proc. Conf. on Uncertainty in Artificial Intelligence (PMLR), 1129–1138.Google Scholar
• Helmer O (1967) Analysis of the future: The delphi method. Technical report, Rand Corp., Santa Monica, CA.Google Scholar
• Hogarth R (1987) Judgement and Choice (Wiley, New York).Google Scholar
• Ibrahim JG, Chen MH, et al. (2000) Power prior distributions for regression models. Statist. Sci. 15(1):46–60.Google Scholar
• Ibrahim JG, Chen MH, Gwon Y, Chen F (2015) The power prior: Theory and applications. Statist. Medicine 34(28):3724–3749.Crossref, Google Scholar
• Jaspersen JG, Montibeller G (2015) Probability elicitation under severe time pressure: A rank-based method. Risk Anal. 35(7):1317–1335.Crossref, Google Scholar
• Jenkinson DJ (2005) The elicitation of probabilities: A review of the statistical literature. BEEP working paper, University of Sheffield, UK.Google Scholar
• Johnson SR, Tomlinson GA, Hawker GA, Granton JT, Feldman BM(2010) Methods to elicit beliefs for bayesian priors: a systematic review. J. Clinical Epidemiology 63(4):355–369.Crossref, Google Scholar
• Jones G, Johnson WO (2014) Prior elicitation: Interactive spreadsheet graphics with sliders can be fun, and informative. Amer. Statist. 68(1):42–51.Crossref, Google Scholar
• Joshi C, Ruggeri F, Wilson SP (2017) Prior robustness for bayesian implementation of the fault tree analysis. IEEE Trans. Reliability 67(1):170–183.Crossref, Google Scholar
• Kadane J, Wolfson LJ (1998) Experiences in elicitation.Statistician 47(1):3–19.Crossref, Google Scholar
• Kadane JB, Dickey JM, Winkler RL, Smith WS, Peters SC (1980) Interactive elicitation of opinion for a normal linear model. J. Amer. Statist. Assoc. 75(372):845–854.Crossref, Google Scholar
• Klayman J, Soll JB, Gonzalez-Vallejo C, Barlas S (1999) Overconfidence: It depends on how, what, and whom you ask. Organ. Behav. Human Decision Processes 79(3):216–247.Crossref, Google Scholar
• Lichtenstein S, Fischhoff B, Phillips LD (1982) Calibration of probabilities: The state of the art to 1980. Kahneman D, Slovic P, Tversky A, eds. Judgment under Uncertainty: Heuristics and Biases (Cambridge University Press, Cambridge), 306–334.Crossref, Google Scholar
• Lindley D (1982) The improvement of probability judgements. J. Royal Statist. Soc. [Ser. A] 145(1):117–126.Crossref, Google Scholar
• Lindley D (1983) Reconciliation of probability distributions. Oper. Res. 31(5):866–880.Link, Google Scholar
• Lindley DV, Singpurwalla ND (1986) Reliability (and fault tree) analysis using expert opinions. J. Amer. Statist. Assoc. 81(393):87–90.Crossref, Google Scholar
• Mellers B, Ungar L, Baron J, Ramos J, Gurcay B, Fincher K, Scott SE, et al. (2014) Psychological strategies for winning a geopolitical forecasting tournament. Psych. Sci. 25(5):1106–1115.Crossref, Google Scholar
• Meyer MA, Booker JM (2001) Eliciting and Analyzing Expert Judgment: A Practical Guide (SIAM, Philadelphia).Crossref, Google Scholar
• Morris PA (1974) Decision analysis expert use. Management Sci. 20(9):1233–1241.Link, Google Scholar
• Morris DE, Oakley JE, Crowe JA (2014) A web-based tool for eliciting probability distributions from experts. Environ. Modeling Software 52:1–4.Crossref, Google Scholar
• Niederberger M, Spranger J (2020) Delphi technique in health sciences: A map. Frontiers Public Health 8:457.Crossref, Google Scholar
• Oakley JE, O’Hagan A (2010) Shelf: The Sheffield Elicitation Framework (version 2.0) (School of Mathematics and Statistics, University of Sheffield, Sheffield, UK).Google Scholar
• O’Hagan A (2019) Expert knowledge elicitation: Subjective but scientific. Amer. Statist. 73(suppl 1):69–81.Crossref, Google Scholar
• O’Hagan A, Buck CE, Daneshkhah A, Eiser JR, Garthwaite PH, Jenkinson DJ, Oakley JE, et al. (2006) Uncertain Judgements: Eliciting Experts’ Probabilities (John Wiley & Sons, New York).Crossref, Google Scholar
• Parducci A (1963) Range-frequency compromise in judgment. Psych. Monographs 77(2):1.Crossref, Google Scholar
• Perälä T, Vanhatalo J, Chrysafi A, et al. (2020) Calibrating expert assessments using hierarchical Gaussian process models. Bayesian Anal. 15(4):1251–1280.Crossref, Google Scholar
• Press SJ (2009) Subjective and Objective Bayesian Statistics: Principles, Models, and Applications, vol. 590 (John Wiley & Sons, New York).Google Scholar
• Ríos Insua D, Ruggeri F (2000) Robust Bayesian Analysis (Springer, New York).Crossref, Google Scholar
• Rougier J, Hill LJ, Sparks RSJ (2013) Risk and Uncertainty Assessment for Natural Hazards (Cambridge University Press).Crossref, Google Scholar
• Russo JE, Schoemaker PJ (1992) Managing overconfidence. Sloan Management Rev. 33(2):7–17.Google Scholar
• Saaty RW (1987) The analytic hierarchy process: What it is and how it is used. Math. Modeling 9(3-5):161–176.Crossref, Google Scholar
• Saaty TL (1988) What is the analytic hierarchy process? Mathematical Models for Decision Support (Springer, Berlin, Heidelberg), 109–121.Google Scholar
• Soll JB, Klayman J (2004) Overconfidence in interval estimates. J. Experiment. Psych. Learn. Memory Cognition 30(2):299.Crossref, Google Scholar
• Spetzler CS, Von Holstein CS (1972) Probability Encoding in Decision Analysis (Stanford Research Institute, Stanford, CA).Google Scholar
• Stefan AM, Evans NJ, Wagenmakers EJ (2020) Practical challenges and methodological flexibility in prior elicitation. Psych. Methods, ePub ahead of print https://doi.org/10.1037/met0000354.Crossref, Google Scholar
• Syed Z, Shabarchin O, Lawryshyn Y (2020) A novel tool for bayesian reliability analysis using ahp as a framework for prior elicitation. J. Loss Prevention Process Industry 64:104024.Crossref, Google Scholar
• Tetlock PE, Gardner D (2016) Superforecasting: The Art and Science of Prediction (Random House).Google Scholar
• Thomas O, Pesonen H, Corander J (2020) Probabilistic elicitation of expert knowledge through assessment of computer simulations. Preprint, submitted March 9, https://arxiv.org/abs/2002.10902.Google Scholar
• Turner JR (2010) New Drug Development: An Introduction to Clinical Trials (Springer Science & Business Media).Crossref, Google Scholar
• Tversky A, Kahneman D (1974) Judgment under uncertainty: Heuristics and biases. Science 185(4157):1124–1131.Crossref, Google Scholar
• Ungar L, Mellers B, Satopää V, Tetlock P, Baron J (2012) The good judgment project: A large scale test of different methods of combining expert predictions. Proc. AAAI Fall Sympos. Machine Aggregation of Human Judgment.Google Scholar
• Wang C, Bier VM (2013) Expert elicitation of adversary preferences using ordinal judgments. Oper. Res. 61(2):372–385.Link, Google Scholar
• Werner C, Bedford T, Cooke RM, Hanea AM, Morales-Napoles O(2017) Expert judgement for dependence in probabilistic modelling: A systematic literature review and future research directions. Eur. J. Oper. Res. 258(3):801–819.Crossref, Google Scholar
• Williams CJ, Wilson KJ, Wilson N (2021) A comparison of prior elicitation aggregation using the classical method and shelf. J. Royal Statist. Soc. Ser. A. 184(3):920–940.Crossref, Google Scholar
• Winkler RL (1967a) The assessment of prior distributions in bayesian analysis. J. Amer. Statist. Assoc. 62(319):776–800.Crossref, Google Scholar
• Winkler RL (1967b) The quantification of judgment: Some methodological suggestions. J. Amer. Statist. Assoc. 62(320):1105–1120.Crossref, Google Scholar
• Winkler RL (1968) The consensus of subjective probability distributions. Management Sci. 15(2):B61–B75.Link, Google Scholar