Identifying Expertise to Extract the Wisdom of Crowds

References

• Armstrong JS (2001) Principles of Forecasting: A Handbook for Researchers and Practitioners (Kluwer Academic, Boston).Crossref, Google Scholar
• Aspinall W (2010) A route to more tractable expert advice. Nature 463(7279):294–295.Crossref, Google Scholar
• Bedford T, Cooke R (2001) Probabilistic Risk Analysis: Foundations and Methods (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• Bettman JR, Luce MF, Payne JW (1998) Constructive consumer choice processes. J. Consumer Res. 25(2):187–217.Crossref, Google Scholar
• Bickel E (2007) Some comparisons among quadratic, spherical, and logarithmic scoring rules. Decision Anal. 4(2):49–65.Link, Google Scholar
• Broomell S, Budescu DV (2009) Why are experts correlated? Decomposing correlations between judges. Psychometrika 74(3):531–553.Crossref, Google Scholar
• Budescu DV (2006) Confidence in aggregation of opinions from multiple sources. Fiedler K, Juslin P, eds. Information Sampling and Adaptive Cognition (Cambridge University Press, Cambridge, UK), 327–352.Google Scholar
• Clemen RT (1989) Combining forecasts: A review and annotated bibliography. Internat. J. Forecasting 5(4):559–609.Crossref, Google Scholar
• Clemen RT (2008) Improving and measuring the effectiveness of decision analysis: Linking decision analysis and behavioral decision research. Kugler T, Smith JC, Connolly T, Son Y-J, eds. Decision Modeling and Behavior in Complex and Uncertain Environments (Springer, New York), 3–31.Crossref, Google Scholar
• Clemen RT, Winkler RL (1986) Combining economic forecasts. J. Bus. Econom. Statist. 4(1):39–46.Crossref, Google Scholar
• Clemen RT, Winkler RL (1999) Combining probability distributions from experts in risk analysis. Risk Anal. 19(2):187–203.Crossref, Google Scholar
• Cooke RM (1991) Experts in Uncertainty (Oxford University Press, Oxford, UK).Google Scholar
• Cooke RM, Goossens LHJ (2008) TU Delft expert judgment data base. Reliability Engrg. System Safety 93(5):657–674.Crossref, Google Scholar
• Davis-Stober CP, Dana J, Budescu DV (2010) A constrained linear estimator for multiple regression. Psychometrika 75(3):521–541.Crossref, Google Scholar
• Davis-Stober CP, Budescu DV, Dana J, Broomell SB (2014) When is a crowd wise? Decision 1(2):79–101.Crossref, Google Scholar
• Dawes RM (1979) The robust beauty of improper linear models in decision making. Amer. Psychologist 34(7):571–582.Crossref, Google Scholar
• de Finetti B (1962) Does it make sense to speak of “Good probability appraisers”? Good IJet al., eds. The Scientist Speculates, An Anthology of Partly-Baked Ideas (Basic Books, New York), 357–364.Google Scholar
• Denrell J, Fang C (2010) Predicting the next big thing: Success as a signal of poor judgment. Management Sci. 56(10):1653–1667.Link, Google Scholar
• Evgeniou T, Fang L, Hogarth RH, Karelaia N (2013) Competitive dynamics in forecasting: The interaction of skill and uncertainty. J. Behavioral Decision Making 26(4):375–384.Crossref, Google Scholar
• French S (1985) Group consensus probability distributions: A critical survey. Bernardo JM, DeGroot MH, Lindley DV, Smith AFM, eds. Bayesian Statistics, Vol. 2 (North-Holland, Amsterdam), 183–201.Google Scholar
• French S (2011) Expert judgment, meta-analysis and participatory risk analysis. Decision Anal. 9(2):119–127.Link, Google Scholar
• Genest C, Zidek JV (1986) Combining probability distributions: A critique and annotated bibliography. Statist. Sci. 1(1):114–148.Crossref, Google Scholar
• Gilovich T, Griffin D, Kahneman D (2002) Heuristics and Biases: The Psychology of Intuitive Judgment (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• Hastie R, Kameda T (2005) The robust beauty of majority rules in group decisions. Psych. Rev. 112(2):494–508.Crossref, Google Scholar
• Herzog SM, Hertwig R (2009) The wisdom of many in one mind: Improving individual judgments with dialectical bootstrapping. Psych. Sci. 20(2):231–237.Crossref, Google Scholar
• Herzog SM, Hertwig R (2011) The wisdom of ignorant crowds: Predicting sport outcomes by mere recognition. Judgment and Decision Making 6(1):58–72.Google Scholar
• Hogarth RM (1978) A note on aggregating opinions. Organ. Behav. Human Performance 21(1):40–46.Crossref, Google Scholar
• Hora SC, Fransen BR, Hawkins N, Susel I (2013) Median aggregation of distribution functions. Decision Anal. 10(4):279–291.Link, Google Scholar
• Jose VRR, Grushka-Cockayne Y, Lichtendahl KC Jr (2014) Trimmed opinion pools and the crowd's calibration problem. Management Sci. 60(2):463–475.Link, Google Scholar
• Kutner MH, Nachtsheim CJ, Neter J, Li W (2005) Applied Linear Statistical Models, 5th ed. (McGraw Hill, Irwin, CA).Google Scholar
• Larrick RP, Mannes AE, Soll JB (2011) The social psychology of the wisdom of crowds. Krueger JI, ed. Frontiers of Social Psychology: Social Judgment and Decision Making (Psychology Press, New York), 227–242.Google Scholar
• Lee MD, Zhang S, Shi J (2011) The wisdom of the crowd playing the price is right. Memory and Cognition 39(5):914–923.Crossref, Google Scholar
• Leonhardt D (2012) When the crowd isn't wise. New York Times (July 7), http://www.nytimes.com/2012/07/08/sunday-review/when-the-crowd-isnt-wise.html.Google Scholar
• Lichtendahl KC Jr, Grushka-Cockayne Y, Pfeifer PE (2013) The wisdom of the competitive crowds. Oper. Res. 61(6):1383–1398.Link, Google Scholar
• Lin S-W, Cheng C-H (2009) The reliability of aggregated probability judgments obtained through Cooke's classical model. J. Model. Management 4(2):149–161.Crossref, Google Scholar
• Lorenz J, Rauhutb H, Schweitzera F, Helbing D (2011) How social influence can undermine the wisdom of crowd effect. Proc. Natl. Acad. Sci. 108(2):9020–9025.Crossref, Google Scholar
• Makridakis S, Winkler RL (1983) Averages of forecasts: Some empirical results. Management Sci. 29(9):987–996.Link, Google Scholar
• Mannes AE, Soll JB, Larrick RP (2014) The wisdom of select crowds. J. Personality Soc. Psych. 107(2):276–299.Crossref, Google Scholar
• Simmons J, Nelson LD, Galak J, Frederick S (2011) Intuitive biases in choice vs. estimation: Implications for the wisdom of crowds. J. Consumer Res. 38(1):1–15.Crossref, Google Scholar
• Soll JB, Larrick RP (2009) Strategies for revising judgment: How (and how well) people use others' opinions. J. Experiment. Psych.: Learn., Memory and Cognition 35(3):780–805.Crossref, Google Scholar
• Sunstein CR (2006) Infotopia: How Many Minds Produce Knowledge (Oxford University Press, New York).Crossref, Google Scholar
• Surowiecki J (2004) The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations (Little, Brown, London).Google Scholar
• Tetlock PE (2005) Expert Political Opinion, How Good Is It? How Can We Know? (Princeton University Press, Princeton, NJ).Google Scholar
• Turner BM, Steyvers M, Merkle EC, Budescu DV, Wallsten TS (2014) Forecast aggregation via recalibration. Machine Learn. 95(3):261–289.Crossref, Google Scholar
• Wallsten TS, Budescu DV (1983) Encoding subjective probabilities: A psychological and psychometric review. Management Sci. 29(2):151–173.Link, Google Scholar
• Wallsten TS, Diederich A (2001) Understanding pooled subjective probability estimates. Math. Soc. Sci. 41(1):1–18.Crossref, Google Scholar
• Wallsten TS, Budescu DV, Zwick R (1993) Comparing the calibration and coherence of numerical and verbal probability judgments. Management Sci. 39(2):176–190.Link, Google Scholar
• Wallsten TS, Budescu DV, Erev I, Diederich A (1997) Evaluating and combining subjective probability estimates. J. Behavioral Decision Making 10(3):243–268.Crossref, Google Scholar
• Wang G, Kulkarni SR, Poor HV, Osherson DN (2011) Aggregating large sets of probabilistic forecasts by weighted coherent adjustment. Decision Anal. 8(2):28–144.Link, Google Scholar