Wisdom/Madness of Crowds and Perils of Point Forecasts

References

• Aspinall WP (2006) Structured elicitation of expert judgment for probabilistic hazard and risk assessment in volcanic eruptions. Statistics in Volcanology (Geological Society, London), 15–30.Crossref, Google Scholar
• Aspinall WP (2010) A route to more tractable expert advice. Nature 463(7279):294–295.Crossref, Google Scholar
• Aspinall WP, Loughlin SC, Michael FV, Miller AD, Norton GE, Rowley KC, Sparks RSJ, Young SR (2002) The Montserrat Volcano Observatory: Its evolution, organization, role and activities. Druitt TH, Kokelaar BP, eds. The Eruption of Soufrière Hills Volcano, Montserrat, from 1995 to 1999 (Geological Society, London).Crossref, Google Scholar
• Bruine de Bruin W, van der Klaauw W, Topa G (2011) Expectations of inflation: The biasing effect of thoughts about specific prices. J. Econom. Psych. 32(5):834–845.Crossref, Google Scholar
• Burgman M, Carr A, Godden L, Gregory R, McBride M, Flander L, Maguire L (2011) Redefining expertise and improving ecological judgment. Conservation Lett. 4(2):81–87.Crossref, Google Scholar
• Colson AR, Cooke RM (2017) Cross validation for the classical model of structured expert judgment. Reliability Engrg. System Safety 163:109–120.Crossref, Google Scholar
• Colson AR, Cooke RM (2018) Expert elicitation: Using the classical model to validate experts’ judgments. Rev. Environ. Econom. Policy 12(1):113–132.Crossref, Google Scholar
• Cooke RM (1991) Experts in Uncertainty: Opinion and Subjective Probability in Science (Oxford University Press, Oxford, UK).Crossref, Google Scholar
• Cooke RM (2022) Averaging quantiles, variance shrinkage and overconfidence. Futures Foresight Sci. 5(1):e139.Crossref, Google Scholar
• Cooke RM (2025) Wisdom or madness; expert data on wisdom of crowds. Abrahamsen EB, Aven T, Bouder F, Flage R, Ylönen M, eds. Proc. 35th Eur. Safety Reliability 33th Soc. Risk Anal. Eur. Conf. (Research Publishing, Singapore).Google Scholar
• Cooke RM, Goossens LHJ (2008) TU Delft expert judgment data base. Reliability Engrg. System Safety 93(5):657–674.Crossref, Google Scholar
• Cooke RM, Marti D, Mazzuchi TA (2021) Expert forecasting with and without uncertainty quantification and weighting: What do the data say? Internat. J. Forecasting 37(1):378–387.Crossref, Google Scholar
• Cooke RM, Nieboer D, Misiewicz J (2014) Fat-Tailed Distributions: Data, Diagnostics and Dependence (Wiley, Hoboken, NJ).Crossref, Google Scholar
• Eggstaff JW, Mazzuchi TA, Sarkani S (2014) The effect of the number of seed variables on the performance of Cooke’s classical model. Reliability Engrg. System Safety 121:72–82.Crossref, Google Scholar
• Embrechts P, Kluppelberg C, Mikosch T (2003) Modelling Extremal Events for Insurance and Finance (Springer–Verlag, Berlin).Google Scholar
• Flores BE (1986) A pragmatic view of accuracy measurement in forecasting. Omega 14(2):93–98.Crossref, Google Scholar
• Galton F (1907) Vox populi. Nature 75:450–451.Crossref, Google Scholar
• Gneiting T, Balabdaoui F, Raftery AE (2007) Probabilistic forecasts, calibration and sharpness. J. R. Statist. Soc. B 69(Part 2):243–368.Crossref, Google Scholar
• Gupta N, Chavan SR (2021) Characterizing the tail behaviour of daily precipitation probability distributions over India using the obesity index. Internat. J. Climatology 42(4):2543–2565.Crossref, Google Scholar
• Hald T, Aspinall W, Devleesschauwer B, Cooke RM, Corrigan T, Havelaar AH, Gibb H, et al. (2016) World Health Organization estimates of the relative contributions of food to the burden of disease due to selected foodborne hazards: A structured expert elicitation. PLoS One 11(1):e0145839.Crossref, Google Scholar
• Hanea AM, McBride MF, Burgman MA, Wintle BC (2018) The value of performance weights and discussion in aggregated expert judgments. Risk Anal. 38(9):1781–1794.Crossref, Google Scholar
• Kendall MG, Kendall SFH, Babington Smith B (1939) The distribution of spearman’s coefficient of rank correlation in a universe in which all rankings occur an equal number of times. Biometrika 30(3/4):251–273.Crossref, Google Scholar
• Lichtendahl KC Jr, Grushka-Cockayne Y, Winkler RL (2013) Is it better to average probabilities or quantiles? Management Sci. 59(7):1594–1611.Link, Google Scholar
• Mackay C (1841) Memoirs of Extraordinary Popular Delusions and the Madness of Crowds, vol. I (Richard Bentley, London).Google Scholar
• Morley SK, Brito TV, Welling DT (2018) Measures of model performance based on the log accuracy ratio. Space Weather 16(1):69–88.Crossref, Google Scholar
• Murray D (2019) The Madness of Crowds: Gender, Race and Identity (Bloomsbury, London).Google Scholar
• Niu X, Harvey N (2022) Point, interval, and density forecasts: Differences in bias, judgment noise, and overall accuracy. Futures Foresight Sci. 4(3–4):e124.Crossref, Google Scholar
• Oreskes N (2019) Why Trust Science? (Princeton University Press, Princeton, NJ).Google Scholar
• Pareto V (1898) Cours d’economie politique. J. Political Econom. 6:549–552.Google Scholar
• Planck MK (1950) Scientific Autobiography and Other Papers (Philosophical Library, New York).Crossref, Google Scholar
• Rennert K, Errickson F, Prest BC, Rennels L, Newell RG, Pizer W, Kingdon C, et al. (2022) Comprehensive evidence implies a higher social cost of CO2. Nature 610(7933):687–692.Crossref, Google Scholar
• Surowieki J (2005) The Wisdom of Crowds, reprint ed. (Anchor, New York).Google Scholar
• Taleb N (2007) The Black Swan: The Impact of the Highly Improbable (Random House, New York).Google Scholar
• Thomas RP, Lawrence A (2018) Assessment of expert performance compared across professional domains. J. Appl. Res. Memory Cognition 7(2):167–176.Crossref, Google Scholar
• Weiss DJ, Shanteau J (2004) The vice of consensus and the virtue of consistency. Smith K, Shanteau J, Johnson P, eds. Psychological Investigations of Competence in Decision Making (Cambridge University Press, Cambridge, UK), 226–240.Google Scholar
• Wietzke LM, Merz B, Gerlitz L, Kreibich H, Guse B, Castellarin A (2020) Comparative analysis of scalar upper tail indicators. Hydrological Sci. J. 65(10):625–1639.Crossref, Google Scholar
• Winkler RL, Grushka-Cockayne Y, Lichtendahl KC Jr, Jose R, Victor R (2019) Probability forecasts and their combination: A research perspective. Decision Anal. 16(4):239–260.Link, Google Scholar
• Xu TL, Rodica Ioana D, Florin S (2025) Identifying fat-tailed distributions: Methods and insights. 10th Internat. Conf. Math. Inform. (Centre for the Study of Complexity, Babeş-Bolyai University, Cluj Napoca, Romania).Google Scholar