From Opacity to Transparency: User Behavior and Downstream Effects in Algorithmic Evaluation

Organizations are increasingly integrating artificial intelligence (AI) into traditionally human-driven evaluation processes, such as recruitment and performance evaluation. Yet, the complexity of the underlying AI algorithms often renders these processes opaque, limiting users’ understanding and potentially inducing stress and other behavioral changes. In response to these challenges, transparency is often advocated as a mitigative approach, but its effects remain somewhat ambiguous. Although transparency in algorithmic evaluation may mitigate interviewees’ stress in the recruitment context, it may, for example, also incentivize opportunistic impression management (IM), thereby engendering a transparency paradox. Focusing on this tension, we theorize that while algorithmic evaluation, relative to human evaluation, may heighten stress and deceptive IM, transparency has the potential to mitigate these disruptions. We test this proposition across eight studies, including two main experiments and a quasi-field replication. In Experiment I, we found that although algorithmic evaluation increased interviewees’ stress and deceptive IM relative to human evaluation, transparency counteracted these effects, aligning them closely with human evaluation. In Experiment II, we study how these effects caused deviations in downstream interview performance when such performance is assessed in human-only, AI-augmented (wherein human decision makers are assisted by AI scores), or fully automated decision-making configurations. Although transparency consistently reduced deviations in stress and IM from human evaluation for interviewees, its corrective effect on interview performance was constrained on the evaluation side. When AI scores were present, evaluators anchored on them, discounting their own judgment, even as the AI scores failed to distinguish honest from deceptive behaviors. Together, these findings disentangle the dual-edged effects of transparency in algorithmic evaluation, showing that its benefits depend not only on how users respond, but also on how evaluation is conducted. This work advances research on the use of AI in decision making and socio-technical systems offering implications for organizations, policymakers, and AI developers.

History: Ahmed Abbasi, Senior Editor; Idris Adjerid, Associate Editor.

Funding: The authors acknowledge the Indiana University Pervasive Technology Institute for providing supercomputing resources that have contributed to the research results reported within this paper.

Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2023.0579.