A Heuristic for Combining Correlated Experts When There Are Few Data

References

• Aiolfi M, Timmermann A (2006) Persistence in forecasting performance and conditional combination strategies. J. Econometrics 135(1–2):31–53.Crossref, Google Scholar
• Armstrong JS (2001) Combining forecasts. Armstrong JS, ed. Principles of Forecasting (Springer, Boston), 417–439.Crossref, Google Scholar
• Barnard J, McCulloch R, Meng X-L (2000) Modeling covariance matrices in terms of standard deviations and correlations, with application to shrinkage. Statist. Sinica 10(4):1281–1311.Google Scholar
• Bates JM, Granger CWJ (1969) The combination of forecasts. Oper. Res. Quart. 20(4):451–468.Crossref, Google Scholar
• Broomell SB, Budescu DV (2009) Why are experts correlated? Decomposing correlations between judges. Psychometrika 74(3):531–553.Crossref, Google Scholar
• Budescu DV, Chen E (2015) Identifying expertise to extract the wisdom of crowds. Management Sci. 61(2):267–280.Link, Google Scholar
• Bunn D (1985) Statistical efficiency in the linear combination of forecasts. Internat. J. Forecast. 1(2):151–163.Crossref, Google Scholar
• Chen E, Budescu DV, Lakshmikanth SK, Mellers BA, Tetlock PE (2016) Validating the contribution-weighted model: Robustness and cost-benefit analyses. Decision Anal. 13(2):128–152.Link, Google Scholar
• Claeskens G, Magnus JR, Vasnev AL, Wang W (2016) The forecast combination puzzle: A simple theoretical explanation. Internat. J. Forecast. 32(3):754–762.Crossref, Google Scholar
• Clemen R (1989) Combining forecasts: A review and annotated bibliography. Internat. J. Forecast. 5(4):559–583.Crossref, Google Scholar
• Clemen R, Winkler R (1986) Combining economic forecasts. J. Bus. Econom. Statist. 4(1):39–46.Crossref, Google Scholar
• Croushore D (1993) Introducing: The survey of professional forecasters. Bus. Rev. (Philadelphia) 3:3–15.Google Scholar
• Davis-Stober CP, Budescu DV, Broomell SB, Dana J (2015) The composition of optimally wise crowds. Decision Anal. 12(3):130–143.Link, Google Scholar
• Davis-Stober CP, Budescu DV, Dana J, Broomell SB (2014) When is a crowd wise? Decision (Wash. D.C.) 1(2):79–101.Google Scholar
• DeGroot MH (2004) Optimal Statistical Decisions. Wiley Classics Library (Wiley-Interscience, Hoboken, NJ).Crossref, Google Scholar
• Diebold FX, Pauly P (1990) The use of prior information in forecast combination. Internat. J. Forecast. 6(4):503–508.Crossref, Google Scholar
• Diebold FX, Shin M (2019) Machine learning for regularized survey forecast combination: Partially-egalitarian lasso and its derivatives. Internat. J. Forecast. 35(4):1679–1691.Crossref, Google Scholar
• Geisser S (1965) A Bayes approach for combining correlated estimates. J. Amer. Statist. Assoc. 60(310):602–607.Crossref, Google Scholar
• Gelfand AE, Smith AFM (1990) Sampling-based approaches to calculating marginal densities. J. Amer. Statist. Assoc. 85(410):398–409.Crossref, Google Scholar
• Genre V, Kenny G, Meyler A, Timmermann A (2013) Combining expert forecasts: Can anything beat the simple average? Internat. J. Forecast. 29(1):108–121.Crossref, Google Scholar
• Gneiting T, Raftery AE (2005) Weather forecasting with ensemble methods. Science 310(5746):248–249.Crossref, Google Scholar
• Graefe A, Armstrong JS, Jones RJ, Cuzan AG (2014) Combining forecasts: An application to elections. Internat. J. Forecast. 30(1):43–54.Crossref, Google Scholar
• Granger CWJ (1989) Invited review combining forecasts—Twenty years later. J. Forecast. 8(3):167–173.Crossref, Google Scholar
• Granger CWJ, Ramanathan R (1984) Improved methods of combining forecasts. J. Forecast. 3(2):197–204.Crossref, Google Scholar
• Gruber MHJ (1998) Improving Efficiency by Shrinkage: The James-Stein and Ridge Regression Estimators (Marcel Dekker, Inc., New York).Google Scholar
• Gunter S (1992) Nonnegativity restricted least squares combinations. Internat. J. Forecast. 8(1):45–59.Crossref, Google Scholar
• Jose VRR, Winkler RL (2008) Simple robust averages of forecasts: Some empirical results. Internat. J. Forecast. 24(1):163–169.Crossref, Google Scholar
• Karvetski CW, Olson KC, Mandel DR, Twardy CR (2013) Probabilistic coherence weighting for optimizing expert forecasts. Decision Anal. 10(4):305–326.Link, Google Scholar
• Larrick RP, Soll JB (2006) Intuitions about combining opinions: Misappreciation of the averaging principle. Management Sci. 52(1):111–127.Link, Google Scholar
• Larrick RP, Burson KA, Soll JB (2007) Social comparison and confidence: When thinking you’re better than average predicts overconfidence (and when it does not). Organ. Behav. Human Decision Processes 102(1):76–94.Crossref, Google Scholar
• MacQueen J (1967) Some methods for classification and analysis of multivariate observations. Le Cam L, Neyman J, eds. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1 (University of California Press, Oakland), 281–297.Google Scholar
• Mannes AE, Soll JB, Larrick RP (2014) The wisdom of select crowds. J. Personality Soc. Psych. 107(2):276–299.Crossref, Google Scholar
• Newbold P, Granger CWJ (1974) Experience with forecasting univariate time series and the combination of forecasts. J. Roy. Statist. Soc. Ser A 137(2):131–165.Crossref, Google Scholar
• Olsson H, Loveday J (2015) A comparison of small crowd selection methods. Noelle DC, Dale R, Warlaumont AS, Yoshimi J, Matlock T, Jennings CD, Maglio PP, eds. Proceedings of the Thirty-Seventh Annual Conference of the Cognitive Science Society (Cognitive Science Society, Austin, TX), 1769–1774.Google Scholar
• Press SJ (1972) Applied Multivariate Analysis (Holt, Rinehart, and Winston, New York).Google Scholar
• Schmittlein DC, Kim J, Morrison DG (1990) Combining forecasts: Operational adjustments to theoretically optimal rules. Management Sci. 36(9):1044–1056.Link, Google Scholar
• Smith J, Wallis KF (2009) A simple explanation of the forecast combination puzzle. Oxford Bull. Econom. Statist. 71(3):331–355.Crossref, Google Scholar
• Soll JB, Larrick RP (2009) Strategies for revising judgment: How (and how well) people use others’ opinions. J. Experiment. Psych. Learn. Mem. Cogn. 35(3):780–805.Crossref, Google Scholar
• Soll JB, Mannes AE (2011) Judgmental aggregation strategies depend on whether the self is involved. Internat. J. Forecast. 27(1):81–102.Crossref, Google Scholar
• Stan Development Team (2022) RStan: The R interface to Stan. https://mc-stan.org/.Google Scholar
• Stock J, Watson M (2004) Combination forecasts of output growth in a seven-country data set. J. Forecast. 23:405–430.Crossref, Google Scholar
• Surowiecki J (2005) The Wisdom of Crowds (Anchor, New York).Google Scholar
• Timmermann A (2006) Forecast combinations. Elliott G, Granger CWJ, Timmermann A, eds. Handbook of Economic Forecasting, vol. 1 (Elsevier, Amsterdam), 135–196.Crossref, Google Scholar
• Winkler R (1981) Combining probability distributions from dependent information sources. Management Sci. 27(4):479–488.Link, Google Scholar
• Winkler R, Clemen R (1992) Sensitivity of weights in combining forecasts. Oper. Res. 40(3):609–614.Link, Google Scholar
• Winkler R, Makridakis S (1983) The combination of forecasts. J. Roy. Statist. Soc. Ser A 146(2):150–157.Crossref, Google Scholar