Enhancing AI Use: How Complementary System Information Drives Delegation Frequency and Effectiveness

For a collaboration between humans and artificial intelligence (AI) to be fruitful, tasks should be allocated based on their complementary capabilities. Prior research shows that when humans are responsible for allocating tasks between themselves and an AI through delegation, they often delegate too infrequently or delegate the wrong tasks, preventing complementary performance gains.We study how different types of AI system information affect both delegation frequency and delegation effectiveness, which capture the extent to which humans can leverage existing complementarities with AI. Specifically, we study ex-ante AI certainty (the AI’s estimated likelihood of being correct) and ex-post AI outcome information (whether the AI was actually correct on a given task). We show experimentally that presenting either AI certainty before or the AI’s outcome after a delegation decision has no or even negative effects on combined human-AI performance. However, providing both AI certainty and AI outcome information leads to increased delegation frequency as well as more effective delegation, ultimately leading to beneficial performance. We find that ex-ante certainty information calibrates users’ expectations about AI performance on the task-instance level, while ex-post outcome information confirms or disconfirms these expectations. This complementary use of AI system information supports more accurate mental models of the AI’s capabilities, reduces unwarranted algorithm aversion and improves appropriate task allocation. Overall, our results show that the effects of AI system information should not be assessed in isolation. While each signal on its own can be uninformative or even harmful, combining them can reverse the potentially harmful individual effects and facilitate effective human-AI collaboration. Our findings have implications for the design of AI systems in collaborative delegation settings, suggesting that carefully designed system information can help users better leverage complementarities with AI.