Extracting the Wisdom of Crowds When Information Is Shared

Using the wisdom of crowds—combining many individual judgments to obtain an aggregate estimate—can be an effective technique for improving judgment accuracy. In practice, however, accuracy is limited by the presence of correlated judgment errors, which often emerge because information is shared. To address this problem, we propose an elicitation procedure in which respondents are asked to provide both their own best judgment and an estimate of the average judgment that will be given by all other respondents. We develop an aggregation method, called pivoting, which separates individual judgments into shared and private information and then recombines these results in the optimal manner. In several studies, we investigate the method and examine the accuracy of the aggregate estimates. Overall, the empirical data suggest that the pivoting method provides an effective judgment aggregation procedure that can significantly outperform the simple crowd average.

Data and the online appendix are available at https://doi.org/10.1287/mnsc.2018.3047.

This paper was accepted by Manel Baucells, decision analysis.