The Crowd Classification Problem: Social Dynamics of Binary-Choice Accuracy

References

• Aggarwal R, Lucey BM (2007) Psychological barriers in gold prices? Rev. Financial Econom. 16(2):217–230.Crossref, Google Scholar
• Almaatouq A, Rahimian MA, Alhajri A (2020a) When social influence promotes the wisdom of crowds. Preprint, submitted June 22, https://arxiv.org/abs/2006.12471.Google Scholar
• Almaatouq A, Yin M, Watts D (2020b) Collective problem-solving of groups across tasks of varying complexity. Preprint, submitted January 29, https://psyarxiv.com/ra9qy.Google Scholar
• Almaatouq A, Becker J, Houghton JP, Paton N, Watts DJ, Whiting ME (2021) Empirica: A virtual lab for high-throughput macro-level experiments. Behav. Res. Methods, ePub ahead of print March 29, https://doi.org/10.3758/s13428-020-01535-9.Google Scholar
• Almaatouq A, Noriega-Campero A, Alotaibi A, Krafft PM, Moussaid M, Pentland A (2020c) Adaptive social networks promote the wisdom of crowds. Proc. Natl. Acad. Sci. USA 117(21):11379–11386.Crossref, Google Scholar
• Amipour P, Gray SA, Jetter AJ, Introne JE, Singer A, Arlinghaus R (2020) Wisdom of stakeholder crowds in complex social–ecological systems. Nature Sustainability 3(3):191–199.Crossref, Google Scholar
• Arlotto A, Chick SE, Gans N (2014) Optimal hiring and retention policies for heterogeneous workers who learn. Management Sci. 60(1):110–129.Link, Google Scholar
• Asch SE (1951) Effects of group pressure upon the modification and distortion of judgments. Guetzkow H, ed. Groups, Leadership, and Men: Research in Human Relations (Carnegie Press, Pittsburgh), 177–190.Google Scholar
• Ashton RH (1986) Combining the judgments of experts: How many and which ones? Organ. Behav. Human Decision Processes 38(3):405–414.Crossref, Google Scholar
• Atanasov P, Rescober P, Stone E, Swift SA, Servan-Schreiber E, Tetlock P, Ungar L, Mellers B (2017) Distilling the wisdom of crowds: Prediction markets vs. prediction polls. Management Sci. 63(3):691–706.Link, Google Scholar
• Bazerman MH, Moore DA (1994) Judgment in Managerial Decision Making (Wiley, New York).Google Scholar
• Bazerman MH, Curhan JR, Moore DA, Valley KL (2000) Negotiation. Annual Rev. Psych. 51(1):279–314.Crossref, Google Scholar
• Becker J, Almaatouq A, Horvat A (2020) Network structures of collective intelligence: The contingent benefits of group discussion. Preprint, submitted September 15, https://arxiv.org/abs/2009.07202.Google Scholar
• Becker J, Brackbill D, Centola D (2017) Network dynamics of social influence in the wisdom of crowds. Proc. Natl. Acad. Sci. USA 114(26):E5070–E5076.Crossref, Google Scholar
• Budescu DV, Chen E (2014) Identifying expertise to extract the wisdom of crowds. Management Sci. 61(2):267–280.Link, Google Scholar
• Burghardt K, Rand W, Girvan M (2019) Inferring models of opinion dynamics from aggregated jury data. PLoS One 14(7):e0218312.Crossref, Google Scholar
• Cohen MD, March JG, Olsen JP (1972) A garbage can model of organizational choice. Admin. Sci. Quart. 17(1):1–25.Crossref, Google Scholar
• Coleman JS (1966) Foundations for a theory of collective decisions. Amer. J. Sociol. 71(6):615–627.Crossref, Google Scholar
• Collins R (2014) Interaction ritual chains and collective effervescence. von Scheve C, Salmella M, eds. Collective Emotions, Series in Affective Science (Oxford University Press, Oxford, UK), 299–311.Google Scholar
• Cowgill B, Zitzewitz E (2015) Corporate prediction markets: Evidence from google, ford, and firm x. Rev. Econom. Stud. 82(4):1309–1341.Google Scholar
• Csaszar FA, Eggers JP (2013) Organizational decision making: An information aggregation view. Management Sci. 59(10):2257–2277.Link, Google Scholar
• Cyert RM, March JG (1963) A Behavioral Theory of the Firm (Prentice-Hall, Englewood Cliffs, NJ).Google Scholar
• Da Z, Huang X (2020) Harnessing the wisdom of crowds. Management Sci. 66(5):1847–1867.Link, Google Scholar
• Dalkey N, Helmer O (1963) An experimental application of the delphi method to the use of experts. Management Sci. 9(3):458–467.Link, Google Scholar
• DeGroot MH (1974) Reaching a consensus. J. Amer. Statist. Assoc. 69(345):118–121.Google Scholar
• Eichler HG, Kong SX, Gerth WC, Mavros P, Jönsson B (2004) Use of cost-effectiveness analysis in health-care resource allocation decision-making: How are cost-effectiveness thresholds expected to emerge? Value Health 7(5):518–528.Crossref, Google Scholar
• Farrell S (2011) Social influence benefits the wisdom of individuals in the crowd. Proc. Natl. Acad. Sci. USA 108(36):E625.Crossref, Google Scholar
• Fisher D, Charkoudian L (2008) Mediation Guide (Community Mediation Maryland).Google Scholar
• Fisher R, Ury WL, Patton B (2011) Getting to Yes: Negotiating Agreement Without Giving In (Penguin).Google Scholar
• Frey V, van de Rijt A (2020) Social influence undermines the wisdom of the crowd in sequential decision making. Management Sci., ePub ahead of print October 8, https://doi.org/10.1287/mnsc.2020.3713.Google Scholar
• Gardikiotis A, Martin R, Hewstone M (2005) Group consensus in social influence: Type of consensus information as a moderator of majority and minority influence. Personality Soc. Psych. Bull. 31(9):1163–1174.Crossref, Google Scholar
• Golub B, Jackson MO (2010) Naive learning in social networks and the wisdom of crowds. Amer. Econom. J. Microeconom. 2(1):112–149.Crossref, Google Scholar
• Gürçay B, Mellers BA, Baron J (2015) The power of social influence on estimation accuracy. J. Behav. Decision Making 28(3):250–261.Crossref, Google Scholar
• Hartnett N, Kennedy R, Sharp B, Greenacre L (2016) Marketers’ intuitions about the sales effectiveness of advertisements. J. Marketing Behav. 2(2–3):177–194.Google Scholar
• Hastie R (1986) Experimental evidence on group accuracy. Grofman B, Owen G, eds. Information Pooling and Group Decision Making, Decision Research, vol. 2 (Emerald, Bingley, UK), 129–157.Google Scholar
• Hastie R, Kameda T (2005) The robust beauty of majority rules in group decisions. Psych. Rev. 112(2):494–508.Crossref, Google Scholar
• Hogarth RM (1978) A note on aggregating opinions. Organ. Behav. Human Performance 21(1):40–46.Crossref, Google Scholar
• Holman D, Chissick C, Totterdell P (2002) The effects of performance monitoring on emotional labor and well-being in call centers. Motivation Emotion 26(1):57–81.Crossref, Google Scholar
• Hong H, Du Q, Wang G, Fan W, Xu D (2016) Crowd wisdom: The impact of opinion diversity and participant independence on crowd performance. 22nd Amer. Conf. Inform. Systems AMCIS2016.Google Scholar
• Jansen WJ, Jin X, de Winter JM (2016) Forecasting and nowcasting real GDP: Comparing statistical models and subjective forecasts. Internat. J. Forecasting 32(2):411–436.Crossref, Google Scholar
• Jayles B, Kim H-r, Escobedo R, Cezera S, Blanchet A, Kameda T, Sire C, Theraulaz G (2017) How social information can improve estimation accuracy in human groups. Proc. Natl. Acad. Sci. USA 114(47):12620–12625.Crossref, Google Scholar
• Jørgensen M (2004) A review of studies on expert estimation of software development effort. J. Systems Software 70(1–2):37–60.Crossref, Google Scholar
• Kao AB, Berdahl AM, Hartnett AT, Lutz MJ, Bak-Coleman JB, Ioannou CC, Giam X, Couzin ID (2018) Counteracting estimation bias and social influence to improve the wisdom of crowds. J. Roy. Soc. Interface 15(141):20180130.Crossref, Google Scholar
• Kinney W, Burgstahler D, Martin R (2002) Earnings surprise “materiality” as measured by stock returns. J. Accounting Res. 40(5):1297–1329.Crossref, Google Scholar
• Koriat A (2012) When are two heads better than one and why? Sci. 336(6079):360–362.Google Scholar
• Krauss C (2008) Oil hits $100 a barrel for the first time. New York Times (January 2), https://www.nytimes.com/2008/01/02/business/02cnd-oil.html.Google Scholar
• Kurvers RH, Herzog SM, Hertwig R, Krause J, Carney PA, Bogart A, Argenziano G, Zalaudek I, Wolf M (2016) Boosting medical diagnostics by pooling independent judgments. Proc. Natl. Acad. Sci. USA 113(31):8777–8782.Crossref, Google Scholar
• Lazer D, Friedman A (2007) The network structure of exploration and exploitation. Admin. Sci. Quart. 52(4):667–694.Crossref, Google Scholar
• Lorenz J, Rauhut H, Schweitzer F, Helbing D (2011) How social influence can undermine the wisdom of crowd effect. Proc. Natl. Acad. Sci. USA 108(22):9020–9025.Crossref, Google Scholar
• Madirolas G, de Polavieja GG (2015) Improving collective estimations using resistance to social influence. PLOS Comput. Biol. 11(11):e1004594.Crossref, Google Scholar
• Malone TW, Bernstein MS (2015) Handbook of Collective Intelligence (MIT Press, Cambridge, MA).Google Scholar
• Mannes AE (2009) Are we wise about the wisdom of crowds? The use of group judgments in belief revision. Management Sci. 55(8):1267–1279.Link, Google Scholar
• Mannes AE, Soll JB, Larrick RP (2014) The wisdom of select crowds. J. Personality Soc. Psych. 107(2):276–299.Crossref, Google Scholar
• March JG (1991) How decisions happen in organizations. Human Comput. Interaction 6(2):95–117.Crossref, Google Scholar
• Medvec VH, Galinsky AD (2005) Putting more on the table: How making multiple offers can increase the final value of the deal. HBS Negotiation Newsletter 8:4–6.Google Scholar
• Minson JA, Mueller JS, Larrick RP (2018) The contingent wisdom of dyads: When discussion enhances vs. undermines the accuracy of collaborative judgments. Management Sci. 64(9):4177–4192.Link, Google Scholar
• Mossel E, Tamuz O (2017) Opinion exchange dynamics. Probab. Surv. 14:155–204.Crossref, Google Scholar
• Moore P (2019) The quantified workplace. Medium (January 14) https://medium.com/field-guide-to-the-future-of-work/the-quantified-workplace-3874b53353d6.Google Scholar
• Navajas J, Niella T, Garbulsky G, Bahrami B, Sigman M (2018) Aggregated knowledge from a small number of debates outperforms the wisdom of large crowds. Nature Human Behav. 2:126–132.Crossref, Google Scholar
• Nofer M, Hinz O (2014) Are crowds on the internet wiser than experts? The case of a stock prediction community. J. Bus. Econom. 84(3):303–338.Crossref, Google Scholar
• Page SE (2007) The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies (Princeton University Press, Princeton, NJ).Google Scholar
• Palley AB, Soll JB (2019) Extracting the wisdom of crowds when information is shared. Management Sci. 65(5):2291–2309.Abstract, Google Scholar
• Parayitam S, Dooley RS (2007) The relationship between conflict and decision outcomes: Moderating effects of cognitive- and affect-based trust in strategic decision-making teams. Internat. J. Conflict Management 18(1):42–73.Crossref, Google Scholar
• Pierce L, Snow DC, McAfee A (2015) Cleaning house: The impact of information technology monitoring on employee theft and productivity. Management Sci. 61(10):2299–2319.Google Scholar
• Rivera LA (2012) Hiring as cultural matching: The case of elite professional service firms. Amer. Sociol. Rev. 77(6):999–1022.Crossref, Google Scholar
• Schnusenberg O (2006) The stock market behaviour prior and subsequent to new highs. Appl. Financial Econom. 16(6):429–438.Crossref, Google Scholar
• Sherif M (1935) A study of some social factors in perception. Arch. Psych. 187:60.Google Scholar
• Shore J, Bernstein E, Lazer D (2015) Facts and figuring: An experimental investigation of network structure and performance in information and solution spaces. Organ. Sci. 26(5):1432–1446.Link, Google Scholar
• Straub VJ, Tsvetkova M, Yasseri T (2020) The cost of coordination can exceed the benefit of collaboration in performing complex tasks. Preprint, submitted September 23, https://arxiv.org/abs/2009.11038a.Google Scholar
• Stroebe W, Nijstad BA, Rietzschel EF (2010) Chapter four—beyond productivity loss in brainstorming groups: The evolution of a question. Adv. Experiment. Soc. Psych. 43:157–203.Crossref, Google Scholar
• Surowiecki J (2004) The Wisdom of Crowds (Anchor Press, New York).Google Scholar
• Ven AHVD, Delbecq AL (1974) The effectiveness of nominal, delphi, and interacting group decision making processes. Acad. Management J. 17(4):605–621.Crossref, Google Scholar
• Wolf M, Krause J, Carney PA, Bogart A, Kurvers RHJM (2015) Collective intelligence meets medical decision-making: The collective outperforms the best radiologist. PLoS One 10(8):e0134269.Crossref, Google Scholar