Is Your Machine Better Than You? You May Never Know

References

• Acemoglu D, Dahleh MA, Lobel I, Ozdaglar A (2011) Bayesian learning in social networks. Rev. Econom. Stud. 78(4):1201–1236.Crossref, Google Scholar
• Ahsen ME, Ayvaci MUS, Raghunathan S (2019) When algorithmic predictions use human-generated data: A bias-aware classification algorithm for breast cancer diagnosis. Inform. Systems Res. 30(1):97–116.Link, Google Scholar
• Alizamir S, de Véricourt F, Sun P (2013) Diagnostic accuracy under congestion. Management Sci. 59(1):157–171.Link, Google Scholar
• Allon G, Drakopoulos K, Manshadi V (2021) Information inundation on platforms and implications. Oper. Res. 69(6):1784–1804.Link, Google Scholar
• Bastani H, Bastani O, Sinchaisri WP (2021a) Improving human decision-making with machine learning. Working paper, The Wharton School, Operations Information and Decisions, Philadelphia. https://parksinchaisri.github.io/files/tips.pdf.Google Scholar
• Bastani H, Bayati M, Khosravi K (2021b) Mostly exploration-free algorithms for contextual bandits. Management Sci. 67(3):1329–1349.Link, Google Scholar
• Bates AS, Margolis PA, Evans AT (1993) Verification bias in pediatric studies evaluating diagnostic tests. J. Pediatrics 122(4):585–590.Crossref, Google Scholar
• Baumeister RF, Bratslavsky E, Finkenauer C, Vohs KD (2001) Bad is stronger than good. Rev. General Psych. 5(4):323–370.Crossref, Google Scholar
• Begg CB, Greenes RA (1983) Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics 39(1):207–215.Crossref, Google Scholar
• Benjamin DJ (2019) Errors in probabilistic reasoning and judgment biases, Chapter 2. Douglas B, DellaVigna S, Laibson D, eds. Handbook of Behavioral Economics: Applications and Foundations 1, vol. 2 (North-Holland, Amsterdam), 69–186.Google Scholar
• Besbes O, Zeevi A (2009) Dynamic pricing without knowing the demand function: Risk bounds and near-optimal algorithms. Oper. Res. 57(6):1407–1420.Link, Google Scholar
• Bordalo P, Gennaioli N, Shleifer A (2012) Salience theory of choice under risk. Quart. J. Econom. 127(3):1243–1285.Crossref, Google Scholar
• Boyacı T, Özer Ö (2010) Information acquisition for capacity planning via pricing and advance selling: When to stop and act? Oper. Res. 58(5):1328–1349.Link, Google Scholar
• Boyacı T, Canyakmaz C, de Véricourt F (2023) Human and machine: The impact of machine input on decision-making under cognitive limitations. Management Sci., ePub ahead of print March 31, https://doi.org/10.1287/mnsc.2023.4744.Google Scholar
• Broemeling LD (2011) Bayesian estimation of combined accuracy for tests with verification bias. Diagnostics (Basel) 1(1):53–76.Crossref, Google Scholar
• Camacho N, Donkers B, Stremersch S (2011) Predictably non-Bayesian: Quantifying salience effects in physician learning about drug quality. Marketing Sci. 30(2):305–320.Link, Google Scholar
• Cheung WC, Simchi-Levi D, Wang H (2017) Dynamic pricing and demand learning with limited price experimentation. Oper. Res. 65(6):1722–1731.Link, Google Scholar
• Coutts A (2019) Good news and bad news are still news: Experimental evidence on belief updating. Experiment. Econom. 22(2):369–395.Crossref, Google Scholar
• Cowgill B (2019) Bias and productivity in humans and machines. Research paper, Columbia Business School, New York.Google Scholar
• Cukier K, Mayer-Schönberger V, de Véricourt F (2022) Framers: Human Advantage in an Age of Technology and Turmoil (Dutton-Penguin Random House, New York).Google Scholar
• Diaconis P, Freedman D (1986) On the consistency of Bayes estimates. Ann. Statist. 14(1):1–26.Crossref, Google Scholar
• Dietvorst BJ, Simmons JP, Massey C (2015) Algorithm aversion: People erroneously avoid algorithms after seeing them err. J. Experiment. Psych. General 144(1):114–126.Crossref, Google Scholar
• Dietvorst BJ, Simmons JP, Massey C (2018) Overcoming algorithm aversion: People will use imperfect algorithms if they can (even slightly) modify them. Management Sci. 64(3):1155–1170.Link, Google Scholar
• Fudenberg D, Romanyuk G, Strack P (2017) Active learning with a misspecified prior. Theoretical Econom. 12(3):1155–1189.Crossref, Google Scholar
• Gaube S, Suresh H, Raue M, Merritt A, Berkowitz SJ, Lermer E, Coughlin JF, Guttag JV, Colak E, Ghassemi M (2021) Do as AI say: Susceptibility in deployment of clinical decision-aids. NPJ Digital Medicine 4(1):1–8.Crossref, Google Scholar
• Grady D (2019) AI took a test to detect lung cancer. It got an A. The New York Times (May 20), 20.Google Scholar
• Greenes RA, Begg CB (1985) Assessment of diagnostic technologies: Methodology for unbiased estimation from samples of selectively verified patients. Investigative Radiology 20(7):751–756.Crossref, Google Scholar
• Guo Y, Zhang C, Yang XJ (2020) Modeling trust dynamics in human-robot teaming: A Bayesian inference approach. Extended Abstracts 2020 CHI Conf. Human Factors Comput. Systems (Association for Computing Machinery, New York), 1–7.Google Scholar
• Gut A (2009) Stopped Random Walks (Springer, New York).Crossref, Google Scholar
• Harrison JM, Keskin NB, Zeevi A (2012) Bayesian dynamic pricing policies: Learning and earning under a binary prior distribution. Management Sci. 58(3):570–586.Link, Google Scholar
• Herrera H, Hörner J (2013) Biased social learning. Games Econom. Behav. 80:131–146.Crossref, Google Scholar
• Hujoel IA, Jansson-Knodell CL, Hujoel PP, Hujoel ML, Choung RS, Murray JA, Rubio-Tapia A (2021) Estimating the impact of verification bias on celiac disease testing. J. Clinical Gastroenterology 55(4):327–334.Crossref, Google Scholar
• Ibrahim R, Kim S-H, Tong J (2021) Eliciting human judgment for prediction algorithms. Management Sci. 67(4):2314–2325.Link, Google Scholar
• Kahneman D (1973) Attention and Effort, vol. 1063 (Prentice-Hall, Hoboken, NJ).Google Scholar
• Keskin NB, Birge JR (2019) Dynamic selling mechanisms for product differentiation and learning. Oper. Res. 67(4):1069–1089.Abstract, Google Scholar
• Kubat M (2017) An Introduction to Machine Learning, vol. 2 (Springer, Cham, Switzerland).Crossref, Google Scholar
• Lebovitz S, Levina N, Lifshitz-Assaf H (2021) Is AI ground truth really “true”? The dangers of training and evaluating AI tools based on experts’ know-what. Management Inform. Systems Quart. 45(3):1501–1525.Crossref, Google Scholar
• Lebovitz S, Lifshitz-Assaf H, Levina N (2022) To engage or not to engage with AI for critical judgments: How professionals deal with opacity when using AI for medical diagnosis. Organ. Sci. 33(1):126–148.Link, Google Scholar
• Lee HCB, Ba S, Li X, Stallaert J (2018) Salience bias in crowdsourcing contests. Inform. Systems Res. 29(2):401–418.Link, Google Scholar
• McKendrick J (2021) AI adoption skyrocketed over the last 18 months. Accessed February 18, 2022, https://hbr.org/2021/09/ai-adoption-skyrocketed-over-the-last-18-months.Google Scholar
• Möbius MM, Niederle M, Niehaus P, Rosenblat TS (2022) Managing self-confidence: Theory and experimental evidence. Management Sci. 68(11):7793–7817.Link, Google Scholar
• Özer Ö, Zheng Y (2018) Trust and trustworthiness. Donohue K, Katok E, Leider S, eds. The Handbook of Behavioral Operations (Wiley, Hoboken, NJ), 489–523.Google Scholar
• Pepe MS (2003) The Statistical Evaluation of Medical Tests for Classification and Prediction (Oxford University Press, New York).Crossref, Google Scholar
• Petscavage JM, Richardson ML, Carr RB (2011) Verification bias: An underrecognized source of error in assessing the efficacy of medical imaging. Acad. Radiology 18(3):343–346.Crossref, Google Scholar
• Puranam P, Tsetlin I (2021) The limits to explainability. Working paper, INSEAD, Singapore. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3914597.Google Scholar
• Ransohoff DF, Feinstein AR (1978) Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. New England J. Medicine 299(17):926–930.Crossref, Google Scholar
• Reardon S (2019) Rise of robot radiologists. Nature 576(7787):S54–S58.Crossref, Google Scholar
• Schwartzstein J (2014) Selective attention and learning. J. Eur. Econom. Assoc. 12(6):1423–1452.Crossref, Google Scholar
• Simon HA (1955) A behavioral model of rational choice. Quart. J. Econom. 69(1):99–118.Crossref, Google Scholar
• Smith L, Sørensen P (2000) Pathological outcomes of observational learning. Econometrica 68(2):371–398.Crossref, Google Scholar
• Soll JB, Mannes AE (2011) Judgmental aggregation strategies depend on whether the self is involved. Internat. J. Forecasting 27(1):81–102.Crossref, Google Scholar
• Stone M (1961) The opinion pool. Ann. Math. Statist. 32(4):1339–1342.Crossref, Google Scholar
• Sun J, Zhang DJ, Hu H, Van Mieghem JA (2021) Predicting human discretion to adjust algorithmic prescription: A large-scale field experiment in warehouse operations. Management Sci. 68(2):846–865.Link, Google Scholar
• Taylor SE, Thompson SC (1982) Stalking the elusive “vividness” effect. Psych. Rev. 89(2):155–181.Crossref, Google Scholar
• Tiefenbeck V, Goette L, Degen K, Tasic V, Fleisch E, Lalive R, Staake T (2018) Overcoming salience bias: How real-time feedback fosters resource conservation. Management Sci. 64(3):1458–1476.Link, Google Scholar
• Tschandl P, Codella N, Akay BN, Argenziano G, Braun RP, Cabo H, Gutman D, et al. (2019) Comparison of the accuracy of human readers vs. machine-learning algorithms for pigmented skin lesion classification: An open, web-based, international, diagnostic study. Lancet Oncology 20(7):938–947.Crossref, Google Scholar
• Van Donselaar KH, Gaur V, Van Woensel T, Broekmeulen RA, Fransoo JC (2010) Ordering behavior in retail stores and implications for automated replenishment. Management Sci. 56(5):766–784.Link, Google Scholar
• Wang C, Zhang C, Yang XJ (2018) Automation reliability and trust: A Bayesian inference approach. Proc. Human Factors Ergonomics Soc. Annual Meeting, vol. 62 (Sage Publications, Los Angeles), 202–206.Google Scholar
• Whiting PF, Rutjes AW, Westwood ME, Mallett S, QUADAS-2 Steering Group (2013) A systematic review classifies sources of bias and variation in diagnostic test accuracy studies. J. Clinical Epidemiology 66(10):1093–1104.Crossref, Google Scholar
• Wiens J, Saria S, Sendak M, Ghassemi M, Liu VX, Doshi-Velez F, Jung K, et al. (2019) Do no harm: A roadmap for responsible machine learning for healthcare. Nature Medicine 25(9):1337–1340.Crossref, Google Scholar
• Zhou X-H (1993) Maximum likelihood estimators of sensitivity and specificity corrected for verification bias. Comm. Statist. Theory Methods 22(11):3177–3198.Crossref, Google Scholar