The Power of Disagreement: A Field Experiment to Investigate Human–Algorithm Collaboration in Loan Evaluations

Human–algorithm collaboration is becoming increasingly prevalent in the economy and society. However, this collaboration is not always fruitful, and in extreme cases, people become human borgs or totally averse to algorithms. The key to collaborative value is whether humans and algorithms can complement each other in decision making, but it is challenging for humans to disagree with algorithmic recommendations at the right time (i.e., to disagree when algorithms are wrong and not disagree when algorithms are right). To understand the centric role of disagreement in human–algorithm collaboration and examine when and how it benefits, we conducted a field experiment in which human evaluators and algorithms worked together to evaluate loan applications under four scenarios, that is, limited/rich information and with/without disclosure of algorithm rationale. Our results show that human–algorithm collaboration decisions outperformed human- or algorithm-only decisions, and these collaborative values varied across the four scenarios. We further propose a theoretical framework for mechanism examination that centers on the formation and effectiveness of disagreement. We validate the framework empirically and come to the following findings: (1) disagreement exhibits a sizable and nonlinear influence on collaborative value, (2) the differences between human evaluators and algorithms in decision making contribute to disagreement but not to collaborative value, (3) algorithm self-contradiction triggers disagreement and helps human evaluators disagree with algorithms at the right time. These findings provide valuable theoretical insights on how collaborative value is achieved and managerial insights on how to manage disagreement in human–algorithm collaboration.

This paper was accepted by Hemant Bhargava, Special Issue on the Human-Algorithm Connection.

Funding: Y. Zhang appreciates the support from the National Natural Science Foundation of China [Grant 72272003].

Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2022.03844.