Expl(AI)ned: The Impact of Explainable Artificial Intelligence on Users’ Information Processing

References

• Abdel-Karim BM, Pfeuffer N, Carl V, Hinz O (2022) How AI-based systems can induce reflections: The case of AI-augmented diagnostic work. Management Inform. Systems Quart. Forthcoming.Google Scholar
• Abdel-Karim BM, Pfeuffer N, Rohde G, Hinz O (2020) How and what can humans learn from being in the loop? German J. Artificial Intelligence 34(2):199–207.Google Scholar
• Agarwal R, Dhar V (2014) Big data, data science, and analytics: The opportunity and challenge for IS research. Inform. Systems Res. 25(3):443–448.Link, Google Scholar
• Agrawal A, Gans JS, Goldfarb A (2019) Exploring the impact of artificial intelligence: Prediction vs. judgment. Inform. Econom. Policy 47:1–6.Crossref, Google Scholar
• Alavi M, Marakas GM, Yoo Y (2002) A comparative study of distributed learning environments on learning outcomes. Inform. Systems Res. 13(4):404–415.Link, Google Scholar
• Ban GY, El Karoui N, Lim AE (2018) Machine learning and portfolio optimization. Management Sci. 64(3):1136–1154.Link, Google Scholar
• Bauer K, Hinz O, van der Aalst W, Weinhardt C (2021) Expl(AI)n it to me: Explainable AI and information systems research. Bus. Inform. Systems Engrg. 63(2):79–82.Crossref, Google Scholar
• Ben-Ner A, Halldorsson F (2010) Trusting and trustworthiness: What are they, how to measure them, and what affects them. J. Econom. Psych. 31(1):64–79.Google Scholar
• Berente N, Gu B, Recker J, Santhanam R (2021) Managing artificial intelligence. Management Inform. Systems Quart. 45(3):1433–1450.Google Scholar
• Berg J, Dickhaut J, McCabe K (1995) Trust, reciprocity, and social history. Games Econom. Behav. 10(1):122–142.Crossref, Google Scholar
• Bhatt U, Xiang A, Sharma S, Weller A, Taly A, Jia Y, Ghosh J, et al. (2020) Explainable machine learning in deployment. Proc. Conf. on Fairness, Accountability, and Transparency (Association for Computing Machinery, New York).Google Scholar
• Brewer WF (1987) Schemas vs. mental models in human memory. Morris P, ed. Modelling Cognition (John Wiley & Sons, Oxford, UK), 187–197.Google Scholar
• Bussone A, Stumpf S, O’Sullivan D (2015) The role of explanations on trust and reliance in clinical decision support systems. Proc. Internat. Conf. on Healthcare Informatics (Institute of Electrical and Electronics Engineers (IEEE), New York).Google Scholar
• Cabral TS (2021) AI and the right to explanation: Three legal bases under the GDPR. Data Protection Artificial Intelligence 13:29–56.Google Scholar
• Cajias M, Freudenreich P, Freudenreich A, Schäfers W (2020) Liquidity and prices: A cluster analysis of the German residential real estate market. J. Bus. Econom. 90(7):1021–1056.Google Scholar
• Case N (2018) How to become a centaur. J. Design Sci. https://jods.mitpress.mit.edu/pub/issue3-case/release/6?version=53b19e72-d43a-4eda-8c48-6ed3cdc03218.Google Scholar
• Castelo N, Bos MW, Lehmann DR (2019) Task-dependent algorithm aversion. J. Marketing Res. 56(5):809–825.Crossref, Google Scholar
• Chatterjee S, Sarker S, Valacich JS (2015) The behavioral roots of information systems security: Exploring key factors related to unethical IT use. J. Management Inform. Systems 31(4):49–87.Crossref, Google Scholar
• DeGroot MH (1974) Reaching a consensus. J. Amer. Statist. Assoc. 69(345):118–121.Crossref, Google Scholar
• Dellermann D, Ebel P, Söllner M, Leimeister JM (2019) Hybrid intelligence. Bus. Inform. Systems Engrg. 61(5):637–643.Crossref, Google Scholar
• Dhaliwal JS, Benbasat I (1996) The use and effects of knowledge-based system explanations: Theoretical foundations and a framework for empirical evaluation. Inform. Systems Res. 7(3):342–362.Link, Google Scholar
• Dietvorst BJ, Simmons JP, Massey C (2015) Algorithm aversion: People erroneously avoid algorithms after seeing them err. J. Experiment. Psych. General 144(1):114–126.Crossref, Google Scholar
• Dietvorst BJ, Simmons JP, Massey C (2018) Overcoming algorithm aversion: People will use imperfect algorithms if they can (even slightly) modify them. Management Sci. 64(3):1155–1170.Link, Google Scholar
• Dijkstra JJ (1999) User agreement with incorrect expert system advice. Behav. Informs. Tech. 18(6):399–411.Crossref, Google Scholar
• Dodge J, Liao QV, Zhang Y, Bellamy RK, Dugan C (2019) Explaining models: An empirical study of how explanations impact fairness judgment. Proc. Internat. Conf. on Intelligent User Interfaces.Google Scholar
• Doshi-Velez F, Kim B (2017) Toward a rigorous science of interpretable machine learning. Preprint, submitted March 2, https://arxiv.org/abs/1702.08608.Google Scholar
• Erlei A, Nekdem F, Meub L, Anand A, Gadiraju U (2020) Impact of algorithmic decision making on human behavior: Evidence from ultimatum bargaining. Proc. AAAI Conf. on Human Comput. and Crowdsourcing.Google Scholar
• EU (2016) Regulation EU 2016/679 of the European Parliament and of the Council of 27 April 2016, article 22. Official J. Eur. Union Law 119:59.Google Scholar
• EU (2021) Proposal for a regulation EU of the European Parliament and of the Council of April 21, 2021, laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain Union legislative acts. Official J. Eur. Union Law 119. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A52021PC0206.Google Scholar
• Festinger L (1962) Cognitive dissonance. Sci. Amer. 207(4):93–106.Crossref, Google Scholar
• Fügener A, Grahl J, Gupta A, Ketter W (2021a) Cognitive challenges in human–artificial intelligence collaboration: Investigating the path toward productive delegation. Inform. Systems Res. 33(2): 678–696.Google Scholar
• Fügener A, Grahl J, Gupta A, Ketter W (2021b) Will humans-in-the-loop become borgs? Merits and pitfalls of working with AI. Management Inform. Systems Quart. 45(3b):1527–1556.Crossref, Google Scholar
• Garreau D, Luxburg U (2020) Explaining the explainer: A first theoretical analysis of LIME. Proc. Internat. Conf. on Artificial Intelligence and Statist.Google Scholar
• Ge R, Zheng Z, Tian X, Liao L (2021) Human–robot interaction: When investors adjust the usage of robo-advisors in peer-to-peer lending. Inform. Systems Res. 32(3):774–785.Link, Google Scholar
• Ghorbani A, Abid A, Zou J (2019) Interpretation of neural networks is fragile. Proc. AAAI Conf. on Artificial Intelligence. 33(1):3681–3688.Google Scholar
• Gilad B, Kaish S, Loeb PD (1987) Cognitive dissonance and utility maximization: A general framework. J. Econom. Behav. Organ. 8(1):61–73.Crossref, Google Scholar
• Goldstein IM, Lawrence J, Miner AS (2017) Human-machine collaboration in cancer and beyond: The centaur care model. JAMA Oncology 3(10):1303–1304.Crossref, Google Scholar
• Goodman B, Flaxman S (2017) European Union regulations on algorithmic decision-making and a “right to explanation”. AI Magazine 38(3):50–57.Crossref, Google Scholar
• Google AI (2019) Responsible AI practices: Interpretability. Accessed March 8, 2022, https://ai.google/responsibilities/responsible-ai-practices/?category=interpretability.Google Scholar
• Gramegna A, Giudici P (2021) SHAP and LIME: An evaluation of discriminative power in credit risk. Frontiers Artificial Intelligence 4:752558.Crossref, Google Scholar
• Gregor S (2006) The nature of theory in information systems. Management Inform. Systems Quart. 30(3):611–642.Crossref, Google Scholar
• Gregor S, Benbasat I (1999) Explanations from intelligent systems: Theoretical foundations and implications for practice. Management Inform. Systems Quart. 23(4):497–530.Crossref, Google Scholar
• Gunning D, Stefik M, Choi J, Miller T, Stumpf S, Yang GZ (2019) XAI—explainable artificial intelligence. Sci. Robot 4(37):eaay7120.Crossref, Google Scholar
• Harmon-Jones EE (2019) Cognitive Dissonance: Reexamining a Pivotal Theory in Psychology (American Psychological Association).Crossref, Google Scholar
• Hemmer P, Schemmer M, Vössing M, Kühl N (2021) Human-AI complementarity in hybrid intelligence systems: A structured literature review. Proc. 28th Pacific Asia Conf. on Inform. Systems.Google Scholar
• Hoffman M, Kahn LB, Li D (2018) Discretion in hiring. Quart. J. Econom. 133(2):765–800.Crossref, Google Scholar
• Holt CA, Smith AM (2009) An update on Bayesian updating. J. Econom. Behav. Organ. 69(2):125–134.Crossref, Google Scholar
• Ji-Ye Mao IB (2000) The use of explanations in knowledge-based systems: Cognitive perspectives and a process-tracing analysis. J. Management Inform. Systems 17(2):153–179.Crossref, Google Scholar
• Johnson-Laird PN, Goodwin GP, Khemlani SS (2017) Mental models and reasoning. The Routledge International Handbook of Thinking and Reasoning (Routledge, Abingdon-on-Thames, UK), 346–365.Google Scholar
• Jones NA, Ross H, Lynam T, Perez P, Leitch A (2011) Mental models: An interdisciplinary synthesis of theory and methods. Ecological Soc. 16(1). https://www.jstor.org/stable/26268859#metadata_info_tab_contents.Google Scholar
• Jussupow E, Benbasat I, Heinzl A (2020) Why are we averse toward algorithms? A comprehensive literature review on algorithm aversion. Proc. Eur. Conf. on Inform. Systems.Google Scholar
• Jussupow E, Spohrer K, Heinzl A, Gawlitza J (2021) Augmenting medical diagnosis decisions? An investigation into physicians’ decision-making process with artificial intelligence. Inform. Systems Res. 32(3):713–735.Link, Google Scholar
• Kahneman D, Sibony O, Sunstein CR (2021) Noise: A Flaw in Human Judgment (Little, Brown).Google Scholar
• Kaur H, Nori H, Jenkins S, Caruana R, Wallach H, Wortman Vaughan J (2020) Interpreting interpretability: Understanding data scientists’ use of interpretability tools for machine learning. Proc. CHI Conf. on Human Factors in Comput. Systems.Google Scholar
• Klayman J (1995) Varieties of confirmation bias. Psych. Learning Motives 32:385–418.Crossref, Google Scholar
• Kleinmuntz B (1990) Why we still use our heads instead of formulas: Toward an integrative approach. Psych. Bull. 107(3):296.Crossref, Google Scholar
• Knobloch-Westerwick S, Meng J (2009) Looking the other way: Selective exposure to attitude-consistent and counterattitudinal political information. Comm. Res. 36(3):426–448.Crossref, Google Scholar
• Koh PW, Liang P (2017) Understanding black-box predictions via influence functions. Proc. Internat. Conf. on Machine Learn.Google Scholar
• Lakkaraju H, Bastani O (2020) “How do I fool you?” Manipulating user trust via misleading black box explanations. Proc. AAAI/ACM Conf. on AI, Ethics, and Society.Google Scholar
• Lakkaraju H, Kamar E, Caruana R, Leskovec J (2019) Faithful and customizable explanations of black box models. Proc. AAAI/ACM Conf. on AI, Ethics, and Society.Google Scholar
• Lim KH, Ward LM, Benbasat I (1997) An empirical study of computer system learning: Comparison of co-discovery and self-discovery methods. Inform. Systems Res. 8(3):254–272.Link, Google Scholar
• Lipton ZC (2018) The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue 16(3):31–57.Crossref, Google Scholar
• Logg JM, Minson JA, Moore DA (2019) Algorithm appreciation: People prefer algorithmic to human judgment. Organ. Behav. Human Decision Processes 151:90–103.Crossref, Google Scholar
• Lu Z, Yin M (2021) Human reliance on machine learning models when performance feedback is limited: Heuristics and risks. Proc. CHI Conf. on Human Factors in Comput. Systems.Google Scholar
• Lundberg SM, Lee SI (2017) A unified approach to interpreting model predictions. Proc. Conf. on Neural Inform. Processing Systems.Google Scholar
• Malle BF (2006) How the Mind Explains Behavior: Folk Explanations, Meaning, and Social Interaction (MIT Press, Cambridge, MA).Google Scholar
• Meske C, Bunde E, Schneider J, Gersch M (2022) Explainable artificial intelligence: Objectives, stakeholders, and future research opportunities. Inform. Systems Management 39(1):53–63.Crossref, Google Scholar
• Meta AI (2021) Facebook’s five pillars of responsible AI. Accessed March 8, 2022, https://ai.facebook.com/blog/facebooks-five-pillars-of-responsible-ai/.Google Scholar
• Miettinen T, Kosfeld M, Fehr E, Weibull J (2020) Revealed preferences in a sequential prisoners’ dilemma: A horse-race between six utility functions. J. Econom. Behav. Organ. 173:1–25.Crossref, Google Scholar
• Molnar C (2020) Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. Accessed January 14, 2022, https://christophm.github.io/interpretable-ml-book.Google Scholar
• Poursabzi-Sangdeh F, Goldstein DG, Hofman JM, Wortman Vaughan JW, Wallach H (2021) Manipulating and measuring model interpretability. Proc. CHI Conf. on Human Factors in Comput. Systems.Google Scholar
• Pyszczynski T, Greenberg J (1987) Toward an integration of cognitive and motivational perspectives on social inference: A biased hypothesis-testing model. Adv. Experiment. Soc. Psych. 20:297–340.Crossref, Google Scholar
• Rabin M, Schrag JL (1999) First impressions matter: A model of confirmatory bias. Quart. J. Econom. 114(1):37–82.Crossref, Google Scholar
• Rader E, Cotter K, Cho J (2018) Explanations as mechanisms for supporting algorithmic transparency. Proc. CHI Conf. on Human Factors in Comput. Systems.Google Scholar
• Rahwan I, Cebrian M, Obradovich N, Bongard J, Bonnefon JF, Breazeal C, Crandall JW, et al. (2019) Machine behaviour. Nature 568(7753):477–486.Crossref, Google Scholar
• Ribeiro MT, Singh S, Guestrin C (2016) “Why should I trust you?” Explaining the predictions of any classifier. Proc. ACM SIGKDD Internat. Conf. on Knowledge Discovery and Data Mining.Google Scholar
• Rico-Juan JR, de La Paz PT (2021) Machine learning with explainability or spatial hedonics tools? An analysis of the asking prices in the housing market in Alicante, Spain. Expert Systems Appl. 171:114590.Crossref, Google Scholar
• Rosenfeld A, Richardson A (2019) Explainability in human–agent systems. Autonomic Agent Multi Agent Systems 33(6):673–705.Crossref, Google Scholar
• Rouse WB, Morris NM (1986) On looking into the black box: Prospects and limits in the search for mental models. Psych. Bull. 100(3):349.Crossref, Google Scholar
• Schanke S, Burtch G, Ray G (2021) Estimating the impact of “humanizing” customer service chatbots. Inform. Systems Res. 32(3):736–751.Link, Google Scholar
• Schön DA (2017) The Reflective Practitioner: How Professionals Think in Action (Routledge, Abingdon-on-Thames, UK).Crossref, Google Scholar
• Senoner J, Netland T, Feuerriegel S (2021) Using explainable artificial intelligence to improve process quality: Evidence from semiconductor manufacturing. Management Sci. 68(8):5704–5723.Google Scholar
• Shapley LS (1953) A value for n-person games. Contributions to the Theory of Games (AM-28), vol. II (Princeton University Press, Princeton, NJ).Crossref, Google Scholar
• Teodorescu MH, Morse L, Awwad Y, Kane GC (2021) Failures of fairness in automation require a deeper understanding of human-ML augmentation. Management Inform. Systems Quart. 45(3b):1483–1499.Crossref, Google Scholar
• Tschandl P, Rinner C, Apalla Z, Argenziano G, Codella N, Halpern A, Janda M, et al. (2020) Human–computer collaboration for skin cancer recognition. Nature Medicine 26(8):1229–1234.Crossref, Google Scholar
• van den Broek E, Sergeeva A, Huysman M (2021) When the machine meets the expert: An ethnography of developing AI for hiring. Management Inform. Systems Quart. 45(3):1557–1580.Crossref, Google Scholar
• Vandenbosch B, Higgins C (1996) Information acquisition and mental models: An investigation into the relationship between behaviour and learning. Inform. Systems Res. 7(2):198–214.Link, Google Scholar
• Vilone G, Longo L (2021) Notions of explainability and evaluation approaches for explainable artificial intelligence. Inform. Fusion 76:89–106.Crossref, Google Scholar
• Wang W, Benbasat I (2007) Recommendation agents for electronic commerce: Effects of explanation facilities on trusting beliefs. J. Management Inform. Systems 23(4):217–246.Crossref, Google Scholar
• Willison R, Warkentin M (2013) Beyond deterrence: An expanded view of employee computer abuse. Management Inform. Systems Quart. 37(1):1–20.Crossref, Google Scholar
• Yang F, Huang Z, Scholtz J, Arendt DL (2020) How do visual explanations foster end users’ appropriate trust in machine learning? Proc. Internat. Conf. on Intelligent User Interfaces.Google Scholar
• Yin D, Mitra S, Zhang H (2016) Research note—When do consumers value positive vs. negative reviews? An empirical investigation of confirmation bias in online word of mouth. Inform. Systems Res. 27(1):131–144.Link, Google Scholar