Enhancing AI Use: How Complementary System Information Drives Delegation Frequency and Effectiveness

1. Introduction

Delegation of tasks is essential for creating an effective work environment in organizations. It involves transferring decision-making autonomy and responsibility, which can improve task performance by leveraging the strengths of different parties. Furthermore, delegation can help free up resources, allowing individuals to engage in other activities (Leana 1986). Because of advancements in autonomous technologies, delegation is no longer limited to human-to-human interactions, but now also includes interactions between humans and artificial intelligence (AI). With its ability to learn from data and act autonomously, AI exhibits agentic qualities that enable the necessary transfer of autonomy and responsibilities inherent in delegation (Baird and Maruping 2021). Examples abound in various domains, from AI scheduling logistics tasks and selecting applicants for job positions to performing financial transactions on behalf of humans (von Krogh 2018).

Despite AI’s capabilities, delegating all tasks to AI (i.e., full automation) is not yet considered feasible or desirable (Fügener et al. 2026). Instead, humans typically decide which tasks to delegate and which tasks to handle themselves, thereby selectively transferring decision-making responsibility to AI (Adam et al. 2024). Although humans excel at certain tasks, AI systems are better suited for others, and neither consistently outperforms the other across all tasks (Wilder et al. 2020). Prior work suggests that such complementarities arise from asymmetries in both capabilities and information between humans and AI (Hemmer et al. 2025). AI systems excel at processing large amounts of data and detecting statistical patterns, whereas humans can draw on contextual knowledge, intuition, and experience (von Krogh 2018). Consequently, humans and AI often make systematically different errors (Geirhos et al. 2021), creating opportunities to improve performance by leveraging their respective strengths.

These complementarities imply a theoretical potential for complementary performance, understood as performance that exceeds the individual capabilities of humans or AI (Bansal et al. 2021, Gonzalez and Heidari 2025, Lu and Zhang 2025). In line with recent work, this theoretical potential can be distinguished from realized complementary performance, which reflects the extent to which these gains are achieved in practice (Hemmer et al. 2025). Empirical evidence suggests that the realization of such gains remains challenging in human-AI collaboration (Vaccaro et al. 2024). When humans are responsible for delegating tasks to AI, combined performance is often inferior to that of AI alone (Zhang et al. 2020, Fügener et al. 2022), indicating that complementarity potential remains unrealized. However, existing research provides limited understanding of how complementarities can be realized in practice (Hemmer et al. 2025). Leveraging complementarities would require two behavioral components when humans, acting as users, allocate tasks between themselves and AI: users must choose to allocate tasks to AI (delegation frequency) and do so in situations where the AI has a comparative performance advantage (delegation effectiveness). However, users often struggle on both dimensions. Delegation frequency tends to be too low (Fügener et al. 2022), and users often do not recognize the task instances for which delegation would improve performance (Zhang et al. 2020). Thus, complementary performance is not an intrinsic property of human-AI collaboration but, rather, a potential outcome that depends on users’ ability to make informed task allocation decisions.

A central barrier to appropriate delegation is that users often lack an accurate mental model of the AI’s capabilities that specifies when the AI is likely to succeed or fail (Bansal et al. 2019a, Zhang et al. 2020). These mental models capture users’ understanding of the AI and, by shaping their expectations about its performance, guide their collaborative behavior with the system (Bansal et al. 2019a). Although prior research argues that providing system information can help users refine these mental models (Bansal et al. 2021, Bauer et al. 2023), it remains unclear which types of information best support such learning and thereby enable effective delegation. We focus on two types of system information that have typically been studied in isolation and primarily with regard to their effects on system use and trust, rather than on their potential to improve users’ understanding of AI or the effectiveness of its use. The first type is ex ante AI certainty, represented by the probability score generated and communicated by the AI system that quantifies how likely its predicted output is to be correct for a given task instance. Currently, the evidence on whether AI certainty increases system use or improves combined performance is mixed (Fügener et al. 2021, Gonzalez and Heidari 2025). The second type is ex post information on AI outcomes, which indicates whether the AI’s prediction for a given task instance was correct or incorrect. Prior work suggests that outcome information can help users understand an AI’s performance patterns (Bansal et al. 2019a), but it may also trigger algorithm aversion if users interpret observed errors as signs of unreliability (Dietvorst et al. 2015). Although both types of information could, in principle, help users identify tasks for which the AI holds a performance advantage, we argue that each alone also highlights the AI’s fallibility, which can reduce users’ willingness to use the system. What remains unknown is whether and how combining AI certainty and AI outcome information may help users develop a more nuanced mental model of AI’s capabilities, thereby supporting more informed delegation behavior.

Our study examines the causal effects of providing AI certainty and AI outcome information, individually and in combination, on users’ delegation frequency and delegation effectiveness. We rely on expectation confirmation theory (Oliver 1980) and situation awareness theory (Endsley 1995) to derive our hypotheses and explain how AI system information shapes users’ mental models and subsequent delegation behavior. We propose that combining both types of AI system information enables valuable human-AI collaboration. The rationale behind this is that although the two pieces of information individually serve as reminders of the AI’s fallibility, when presented together, they offer users complementary information about the AI’s performance limitations, thereby improving their mental model of the system and enabling more informed delegation decisions. We study this in a context with existing complementarities to assess whether delegation from humans to AI can serve as a behavioral mechanism to achieve complementary performance. Consequently, we examine both whether users choose to delegate (delegation frequency) and whether they do so appropriately (delegation effectiveness). To our knowledge, no prior research has examined the causal effects of presenting AI certainty and/or AI outcome information on the realization of complementarities. We pose the following research question:

To answer this question, we conduct an online experiment (n = 560) in which subjects perform an image classification task. We apply a 2$\times$2 between-subjects design, manipulating whether subjects see the AI’s certainty before making a delegation decision and whether they receive information on the AI’s outcome afterward. Our findings reveal that these two pieces of AI information have contradictory effects when presented on their own. Whereas displaying only AI outcomes negatively impacts delegation frequency and effectiveness, providing only AI certainty increases delegation frequency but does not lead to more effective delegation. Only when AI certainty and AI outcomes are presented together do delegation frequency and delegation effectiveness increase, leading to complementary performance that surpasses the individual performance of both humans and AI. We find that an increase in delegation quality rather than mere quantity (i.e., delegation frequency) drives this increase in performance.

In general, our study advances the literature on delegation to AI, a collaborative setting that has received much less attention than advice taking (Baird and Maruping 2021). Prior work has predominantly examined AI system information in advice contexts, where users retain final responsibility and can inspect AI recommendations before acting (e.g., Rechkemmer and Yin 2022). Delegation, by contrast, requires users to decide ex ante whether to hand over responsibility without seeing the AI’s decision, making accurate mental models even more critical. Building on this distinction, we make three key contributions to the literature on human-AI collaboration and delegation. First, we show that the effects of presenting AI certainty and AI outcome information cannot be understood in isolation. Each signal alone produces limited or even counterproductive effects, but together, they jointly improve users’ expectations about AI performance and mental models, thereby supporting more appropriate delegation decisions. Prior work has typically examined only one type of system information (You et al. 2022) or conflated them (Zhang et al. 2020) and often focused on outcomes such as willingness to use AI (Liu et al. 2025), general system use (Fügener et al. 2021), or trust (Zhang et al. 2020, Rechkemmer and Yin 2022), which do not reveal whether users’ collaborative decisions were appropriate. By unpacking why each signal alone fails to support effective delegation and why only their combination succeeds, we identify mechanisms underlying previously mixed findings on their effects. Second, we show how complementarity potential can be translated into realized complementary performance when delegation decisions are made by users. Whereas prior work often establishes only that human-AI teams outperform humans alone, we assess whether combined performance exceeds that of both humans and AI individually, allowing us to determine whether genuine complementarity is achieved (Hemmer et al. 2025). Rather than characterizing the capability differences or information asymmetries that give rise to complementarity, we focus on whether such complementarity is realized in outcomes when task allocation decisions are made by users. We demonstrate that when users themselves allocate tasks, such complementarities are not realized through a mere increase in delegation frequency, but require users to identify the task instances in which the AI holds a comparative performance advantage and to allocate accordingly. Previous studies have achieved complementary performance in delegation settings only when AI delegated tasks to the user (Fügener et al. 2022) or when users followed predefined rules (Lai et al. 2022). We extend this literature by showing that users can realize complementarities through effective delegation, even when task allocation is not externally optimized but, rather, determined by user behavior. To our knowledge, we are the first to demonstrate complementary performance in a setting where humans themselves delegate to AI. Third, we advance research on algorithm aversion by showing that providing ex ante certainty can mitigate unwarranted reductions in system use after observing imperfect outcomes. Specifically, we identify expectation calibration as a mechanism through which ex ante system information prevents the typical declines in use that follow error observations. This extends prior work by showing that declines in use can be mitigated before errors occur, extending earlier approaches focused on post hoc adjustments (Dietvorst et al. 2018).

The remainder of this paper is structured as follows. In Section 2, we provide an overview of the theoretical background and review the related literature. In Section 3, we derive our hypotheses from the literature. Section 4 provides an overview of our experimental design. Section 5 summarizes our main results, and in Section 6 we present results from additional robustness checks. Lastly, we conclude with a discussion and outlook on future work in Section 7.

2. Theoretical Background and Related Literature

The body of research on human-AI collaboration is rapidly expanding. However, most studies have focused on AI-advised decision making, where users receive recommendations from an AI system but ultimately retain decision responsibility and autonomy (e.g., Jussupow et al. 2021). In such settings, users can inspect and verify the AI’s suggestion before acting. Unlike advice taking, delegation involves transferring decision autonomy from the user to the AI. This creates a higher degree of information asymmetry, because users do not know the AI’s decision in advance (Baird and Maruping 2021). Our research focuses on such delegation environments, because it is becoming increasingly common to delegate and automate tasks (Adam et al. 2023). Although delegation and advice taking share the fundamental challenge of assessing the costs and benefits of relying on AI (Baird and Maruping 2021), delegation presents unique complexities due to the shift in decision-making responsibility. In this section, we include literature on both AI delegation and advice taking to inform our theoretical background.

A central objective in human-AI collaboration is to leverage complementary capabilities between humans and AI. Recent conceptual work distinguishes between different types of complementarity that are relevant for understanding how these opportunities can be realized. Hemmer et al. (2025) differentiate between inherent complementarity, where performance gains on each task instance can be attributed to the individually more accurate agent, and collaborative complementarity, which yields a combined decision that neither party reached individually. Relatedly, Fügener et al. (2026), in workflow terms, contrast between-task complementarity, realized through task allocation, with within-task complementarity, realized through joint work on the same task. In a delegation setting, each task is performed by a single agent rather than jointly. Because the user either retains their own decision or adopts the AI’s, no integrated decision can emerge, meaning that delegation can realize only inherent, between-task complementarity, not collaborative or within-task complementarity. Our study therefore focuses on the former. Realizing such complementarities requires cognitive alignment between humans and AI, including the ability of users to understand and anticipate the strengths and weaknesses of the system (Gonzalez and Heidari 2025). Related research has examined how explanations can support users by helping them to interpret AI decisions (Bansal et al. 2021). We build on this literature by focusing on AI system information, specifically AI certainty and AI outcome information, as a means to support mental model formation. As discussed in Section 2.2, this information can help reduce information asymmetries between users and AI, and Section 2.3 emphasizes that accurate mental models are critical to determine when the use of AI will be beneficial. Integrating these insights, we argue that AI system information can be an important lever in mental model formation, through which users can recognize and act on comparative performance advantages of the AI, leading to improved combined performance.

2.1. Delegation in the Organizational and Information Systems Context

Delegation is grounded in the theory of agency, which examines how one party (the principal) transfers work to another (the agent) (Eisenhardt 1989). This transfer includes risk, because it can be difficult for the principal to verify both the competence of the agent and whether the task was completed as desired (Eisenhardt 1989). To minimize risk, principals delegate more frequently when they perceive agents as capable (Leana 1986) and trustworthy (Aggarwal and Mazumdar 2008). Research on delegation in decision making can be found in various disciplines, including management and leadership (Leana 1986) or psychology (Aggarwal and Mazumdar 2008). They all share a consistent theme of the allocation of rights and responsibilities between principals and agents (Baird and Maruping 2021).

In modern work environments, delegation extends beyond human-to-human interactions, because AI-based systems can also be granted agency. This enables humans to delegate a variety of tasks to increasingly autonomous systems. However, in the information systems (IS) literature, the concept of delegation has often been studied implicitly (Baird and Maruping 2021). Some recent exceptions study delegation to AI at different levels of responsibility, including complete automation of tasks (Ge et al. 2021) and selective delegation based on task difficulty (Lai et al. 2022, Adam et al. 2023). At the task-instance level, research has begun to explore how users decide whether to delegate individual decisions to AI. Fügener et al. (2022) experimentally analyzed delegation both from users to AI and from AI to users in a setting with complementary skills using an image classification task. They find that combined performance was inferior to that of AI alone when users were in charge of delegation. In a separate robustness check, they did not find differences in delegation behavior or performance when users received continuous feedback on their own and AI’s outcomes. In a setting without complementary skills, Zhang et al. (2020) studied the effect of showing certainty scores for a particular AI prediction. Delegation frequency increased, but they found no effect on combined performance. Liu et al. (2025) show that managers’ responses to AI performance feedback are dependent on their willingness to delegate. Although this setting does not involve human-AI complementarities, it reinforces the broader challenge that humans often fail to update their delegation decisions appropriately, even when performance evidence is available. Together, these studies suggest that simply increasing delegation frequency does not necessarily translate into complementary performance. Instead, the core challenge is whether users can delegate the right tasks. Although Fügener et al. (2022) demonstrate that identifying one’s own capabilities is essential for delegating the right tasks, their study design does not allow statements about performance improvement at the task-instance level. Other studies on delegation to AI have not focused on the importance of delegation quality (Zhang et al. 2020, Pinski et al. 2023). To address this gap, our research examines both delegation frequency and delegation effectiveness, focusing on whether a delegation decision leads to complementary performance improvements.

2.2. (Effective) System Use and AI System Information

System use refers to the extent to which users engage with an information system (Burton-Jones and Grange 2013), including through delegation. A key challenge in AI system use is the information asymmetry between users and the system (Wang and Benbasat 2008) that prevents users from understanding when the AI can provide complementary performance benefits. Increasing system transparency by providing information on the system’s structure, procedure, decision rationale, or performance can help reduce this asymmetry (Lehmann et al. 2022, You et al. 2022). Research suggests that detailed explanations of AI systems can enhance user understanding and trust, though their effectiveness may depend on algorithm complexity (Lehmann et al. 2022) or user characteristics (Bolton et al. 2022). However, most modern AI models are complex and opaque, often requiring post hoc explanation techniques that demand additional programming and implementation effort (Liao and Varshney 2022). Therefore, instead of focusing on AI explanations, we study types of AI system information that are readily available in many AI systems—namely, the AI’s certainty and its outcome correctness. Most AI-based systems can provide certainty scores, which indicate the ex ante probability that a system’s decision is correct (Bansal et al. 2021). However, the impact of certainty scores on system use remains unclear. Some studies conducted in algorithm-advised decision-making settings find that certainty scores do not affect users’ acceptance of a system’s recommendation (Cramer et al. 2008) or the willingness to follow a system (Rechkemmer and Yin 2022). Others report that providing AI certainty reduces reliance, leading to lower system use compared with a setting without certainty scores (Fügener et al. 2021). In addition to ex ante certainty information, AI systems can also provide ex post outcome information, informing users whether past AI decisions were correct. The effect of seeing AI outcomes in advice settings depends on the system’s performance: observing correct AI decisions can increase system use (Ahn et al. 2021), whereas observing errors can lead to a persistent reduction of use, even when the AI remains highly accurate overall (Dietvorst et al. 2015, Ahn et al. 2021).

Although research suggests that increased transparency may either promote or decrease system use, this does not determine whether the outcome of usage is beneficial (Bansal et al. 2021). The goal of a collaboration is not simply to maximize system use, but to ensure that users rely on the system for appropriate tasks—that is, using AI when it is correct and reducing system use when it is incorrect (Jussupow et al. 2021). Therefore, effective system use goes beyond mere usage and includes achieving the intended goals with the system (Burton-Jones and Straub 2006, Burton-Jones and Grange 2013). Effective use is often operationalized through assessing the performance of system use (Burton-Jones and Grange 2013). If humans and AI collaborate effectively, they could achieve performance levels beyond those of either working alone. However, if transparency measures fail to convey system reliability accurately, users may over-rely on AI in inappropriate situations or avoid AI assistance when it would be beneficial, leading to suboptimal collaboration (Hemmer et al. 2022).

2.3. Mental Models

We build on the theory of mental models, which describes the internal representations of the external reality humans use to interact with their environment, including objects and systems (Johnson-Laird 1983). When performing tasks, humans rely on relevant mental models to inform their information processing, form expectations, and make decisions (Bauer et al. 2023). Through beliefs and expectations about a system, mental models guide future use of a system (Norman 1987). The process of building and maintaining mental models can be stimulated by the degree of transparency provided by the system (Vandenbosch and Higgins 1996).

In AI-advised decision making, Bansal et al. (2019b) emphasized that users need accurate mental models of AI capabilities to use the system appropriately. The authors state that knowing an AI’s error boundaries is vital for users to form an accurate mental model, which manifests in users relying on the AI more on the instances where it is in fact correct (Bansal et al. 2019b). Error boundaries refer to the separating planes in a decision space that separate the cases for which the AI is correct from those for which it is incorrect (Inkpen et al. 2023). Accurate mental models thus allow appropriate system reliance (Bansal et al. 2019b, Inkpen et al. 2023), which is a premise for effective system use (Endsley 1995). A mismatch between the mental model and the true error boundaries can lead to inappropriate reliance and decreased combined performance (Bansal et al. 2019a, b). The only paper we are aware of studying mental model formation in an AI delegation context is Zhang et al. (2020). They find that presenting AI certainty with a prediction can impact the formation and update of mental models. However, their intervention did not lead to improved combined performance because it was conducted in a setting where complementary skills between users and AI did not exist.

Building on this understanding, we argue that AI system information influences delegation behavior by shaping users’ mental models of the AI’s capabilities. Because delegation decisions are influenced by both general expectations about the AI’s reliability and context-specific judgments about when the AI is likely to be correct, changes in mental models can operate at two levels. To explain these two modes of mental model adjustment, we draw on expectation confirmation theory, which describes how users’ expectations, and thus the mental models that underlie them, are updated based on perceived system performance (Oliver 1980, Grimes et al. 2021), and on situation awareness theory, which explains how users apply and refine their mental models in a concrete task context to guide more accurate reliance decisions (Endsley 1995, Chen and Barnes 2015). These theoretical insights provide the foundation for understanding how AI certainty and AI outcome information should influence users’ delegation behavior. In the following, we develop hypotheses linking AI system information to delegation frequency and delegation effectiveness.

3. Hypotheses Development

We develop hypotheses on how providing two types of AI system information—namely, AI certainty (ex ante information about the AI’s estimated likelihood of being correct) and AI outcome (ex post information about whether the AI outcome was correct)—influence users’ delegation decisions to AI. A delegation decision refers to the user’s choice of whether to delegate a specific task instance to the AI. For each task instance, the user can either complete the task themselves or delegate it to the AI without knowing its prediction in advance (Baird and Maruping 2021). To realize complementary performance, two behavioral dimensions of delegation matter, but in different ways. Delegation frequency determines whether users provide the AI with sufficient opportunities to contribute, whereas delegation effectiveness determines whether these opportunities translate into performance gains. Thus, delegation frequency is a necessary precondition for complementarity, whereas delegation effectiveness ultimately determines whether complementarity is realized. We examine how providing AI certainty and AI outcome information shapes these two dimensions and develop hypotheses accordingly. We preregistered our hypotheses, along with the experimental procedure, at the Open Science Framework (Foster and Deardorff 2017).¹

3.1. Delegation Frequency

We base our hypotheses on delegation frequency on expectation confirmation theory (Oliver 1980), which is used to explain system use in information systems research (Bhattacherjee 2001). Expectation confirmation theory posits that users form a priori expectations about a system’s performance and evaluate these against their perceived experience during system use. This evaluation leads to either confirmation (performance meets or exceeds expectations) or disconfirmation (performance is beneath expectations), which then shapes subsequent decisions on system use (Venkatesh and Goyal 2010, Brown et al. 2014). In our setting, delegation frequency represents such system use. A confirmation of users’ expectations should therefore support continued or increased delegation, whereas disconfirmation should decrease it. Because expectations are grounded in users’ mental models of the AI’s capabilities, the confirmation process also acts as a feedback mechanism through which these mental models are updated based on observed performance (Grimes et al. 2021). Forming accurate expectations is particularly challenging in the context of AI, where system capabilities vary widely and are difficult for users to infer (Grimes et al. 2021). As a result, users often have overly optimistic expectations, assuming near-perfect system performance (Dzindolet et al. 2003, Dietvorst et al. 2015). When they observe information reflecting the AI’s performance, we expect them to compare these against their a priori expectations, resulting in either confirmation or disconfirmation (Brown et al. 2014, Grimes et al. 2021), which, in turn, shapes their delegation frequency.

3.1.1. AI Outcome.

For settings where complementary performance is possible, AI performance cannot be perfect, because it would otherwise outperform the user on all task instances, leaving no potential for improvement through collaboration with humans. Thus, providing users with AI outcome information means that they will inevitably observe errors made by the AI. From an expectation confirmation perspective, these observed errors are likely to disconfirm users’ a priori expectations of near-perfect AI performance (Dzindolet et al. 2003, Dietvorst et al. 2015). When users repeatedly encounter errors in an imperfect AI, their experience no longer matches their initial expectation of a highly accurate AI system, leading to reduced willingness to use the AI—that is, a decreased delegation frequency (Bhattacherjee 2001, Brown et al. 2014). Moreover, expectation confirmation theory suggests that unexpected negative outcomes tend to be particularly salient in shaping subsequent behavior (Oliver 1980). Thus, AI errors would weigh more heavily in users’ evaluations than correct predictions, even when the latter are more frequent. This can distort users’ perception of the AI’s overall reliability and lead them to underuse the system relative to its actual capabilities. Consequently, we expect that AI outcome information will, on average, reduce users’ willingness to delegate to an imperfect AI.

3.1.2. AI Certainty.

From an expectation confirmation perspective, the communicated AI certainty could influence initial user expectations in two ways. First, because certainty values are typically below 100%, they explicitly highlight that the AI is fallible. For users who enter the interaction with overly optimistic expectations assuming near-perfect AI performance, being confronted with probabilities substantially below 100% can already represent a disconfirmation of expectations. Second, without accompanying outcome information, users have no systematic way to assess whether the certainty information is accurate. Because expectation confirmation theory emphasizes that uncertainty about performance and potential disconfirmation of expectations reduce continued system use (Bhattacherjee 2001), we argue that presenting AI certainty alone is unlikely to increase delegation frequency in an imperfect AI setting. Instead, repeatedly seeing that the AI is less than fully certain, without being able to verify whether this uncertainty is justified, may prompt more cautious behavior and a general reluctance to delegate. We therefore hypothesize:

3.1.3. AI Certainty $\times$ AI Outcome.

Although AI outcome and AI certainty can each provide information about the AI’s performance, they do so in different ways: AI outcome reveals realized correctness ex post, whereas AI certainty shapes expectations ex ante. Expectation confirmation theory suggests that it is precisely the alignment between expectations and experience that matters for continued use (Venkatesh and Goyal 2010, Brown et al. 2014). Therefore, when users receive both types of AI system information, we expect them to form expectations based on the communicated AI certainty score and then assess, via AI outcome, whether these expectations are confirmed or disconfirmed. When AI certainty scores reflect the AI’s actual performance (i.e., high certainty is typically followed by correct outcomes and low certainty more often by errors), then the combination of certainty and outcome should enable users to learn if the certainty information is truthful. Over repeated interactions, users can observe that high-certainty predictions tend to be reliable, which confirms their expectations and supports continued delegation; conversely, they may learn to avoid delegating in low-certainty situations. In terms of expectation confirmation theory, outcome verification allows users to confirm or update the expectations formed from communicated certainty, gradually aligning their mental model of the AI’s capabilities with its actual performance (Grimes et al. 2021). We posit that this expectation confirmation process has two implications for delegation frequency. First, it can mitigate the expected negative effect of observing errors alone, because users can attribute some errors to situations where the AI itself signaled low certainty. Second, it can prevent the expected caution induced by AI certainty alone, because users see that high-certainty predictions are often correct and thus worth delegating. As a result, the joint presence of AI certainty and AI outcome verification should, on average, lead to higher delegation frequency than adding the effects of either type of information alone. We therefore posit:

3.2. Delegation Effectiveness

Effective delegation requires users to anticipate whether the AI is more capable than themselves on a given task instance. Because users cannot observe the AI’s prediction before delegating, they must rely on their mental model of the AI to judge whether delegating will improve performance. To theorize how different types of AI system information may affect delegation effectiveness, we draw on situation awareness theory (Endsley 1995), which explains why accurate mental models are essential for effective decision making. Achieving situation awareness involves three levels: users must not only perceive relevant system information (level 1), but also comprehend what it implies about the system’s capabilities in the current context (level 2) and project whether using the system will lead to a better outcome (level 3) (Endsley 1995, Chen and Barnes 2015). As users encounter new or conflicting information, their mental models are updated to maintain accurate situation awareness (Endsley 1995). Following situation awareness theory, effective delegation requires users to develop situation awareness consisting of all three levels. We assume that the first level, perception of relevant information, is fulfilled in all settings we consider.

3.2.1. AI Outcome.

When users see the AI’s outcome after their delegation decisions, they gain a better understanding of past situations, allowing them to assess whether the AI was correct (level 2: comprehension). However, AI outcome information alone does not help users predict the AI performance (level 3: projection), meaning that full situation awareness is not achieved (Chen and Barnes 2015). Without the ability to anticipate future AI performance, users may struggle to make more informed delegation decisions. As a result, the expected reduction in delegation frequency may be uninformed and result in users missing opportunities to delegate tasks where the AI has a complementary performance advantage. When users focus primarily on past AI errors without developing an understanding of when the AI is likely to perform well, they are unable to delegate selectively in ways that enhance joint performance, thereby failing to adjust their mental model to the context and remaining unable to exploit complementarities. Hence, we propose that users reduce delegation frequency without gaining the ability to make more informed decisions about the AI’s likelihood of being correct in future tasks. Instead, they tend to place excessive emphasis on mistakes previously made by the system. We hypothesize:

3.2.2. AI Certainty.

When users see AI certainty before a delegation decision, they could theoretically gain insight into the AI’s expected performance for the current task instance (level 3: projection). Intuitively, this should help them delegate more effectively. However, without additional AI outcome information, the AI certainty score may remain an abstract numerical value that does not convey useful information about the AI’s actual performance and therefore does not enable users to comprehend its meaning in context (level 2: comprehension) (Chen and Barnes 2015). This aligns with evidence that humans frequently struggle to interpret probabilities correctly (Peters et al. 2006), particularly in the context of system-confidence displays (Ma et al. 2023). If users cannot accurately interpret AI certainty information, it does not improve their situation awareness and is unlikely to improve their delegation decisions. As a result, the expected decrease in delegation frequency (Hypothesis 2(a)) occurs without a corresponding improvement in understanding the AI’s capabilities, which, in turn, should lead to reduced delegation effectiveness. Therefore, we hypothesize:

3.2.3. AI Certainty $\times$ AI Outcome.

Providing users with both AI certainty before and AI outcomes after delegation decisions should address the limitations of presenting either type of information alone by enabling all three levels of situation awareness. Specifically, presenting AI outcomes supports comprehension (level 2), whereas AI certainty facilitates projection (level 3) (Chen and Barnes 2015). Users first perceive the certainty information, which provides an initial indication of the AI’s performance state. Following the principles of situation awareness theory, users then integrate this initial information into their comprehension of the AI’s capabilities (Endsley 1995). Observing actual AI outcomes after delegation decisions allows them to verify or refine their comprehension, ultimately improving their ability to accurately project the AI’s future performance. Together, AI certainty and AI outcome should create direct feedback loops that help users interpret probabilistic certainty values and enable the adjustment of users’ mental models of AI performance to the specific context. As mental models become more accurate, users can more effectively predict future AI performance, leading to improved delegation effectiveness and enabling realization of complementarities with AI. Hence, we hypothesize:

4. Experimental Study

4.1. Experimental Task

We employed an image classification task for the experiment, as illustrated in Figure 1. The setup was similar to the experiments in Fügener et al. (2022), with modifications to the images and task structure. Subjects were required to classify a given focal image into 1 of 10 possible classes without seeing the AI’s decision. These classes represent content groups encompassing a wide range of humans, animals, and objects. We selected this task for two key reasons. First, image classification is a domain in which humans and modern AI systems exhibit complementary capabilities. Although humans are naturally good at recognizing objects, deep learning models often match or exceed human accuracy on certain images while performing worse on others (Russakovsky et al. 2015). Importantly, these models do not mirror human decision rules but instead generate their own representational logic through large-scale training data, potentially allowing them to succeed on instances that humans find difficult (Fügener et al. 2021). This self-learned decision logic is fundamental to the type of complementarity examined in our study, because it does not rely on human-specified rules and, therefore, creates genuinely distinct strengths between humans and AI. Second, we aimed for a generalizable experimental setting to generate findings that can be extrapolated across various tasks. Findings from such general tasks are likely to be applicable to specialized tasks, whereas results from specialized tasks may not necessarily generalize to other settings.

An overview of the experimental flow is provided in Figure 2. Before proceeding to the main task, subjects received general instructions and completed an attention check to confirm that they were human. Afterward, they received detailed instructions on the experimental task and were required to complete a practice round to familiarize themselves with the task. Subjects classified 100 images into 1 of 10 potential classes sourced from the ImageNet database (https://www.image-net.org). Each subject classified the same 100 images in randomized order. For each potential class, we displayed 13 exemplary images and the class name (similar to Russakovsky et al. 2015). A classification was considered correct if the focal image was assigned to the correct class as defined by the data set’s ground truth. Subjects first classified each image on their own (without AI support) and stated their certainty about their own classification on a scale from 0% (uncertain) to 100% (certain). After submitting each classification, subjects were directed to the next screen displaying the option to delegate the individual classification decision to the AI. This delegation screen displayed the previously classified image and allowed participants to either keep their own answer or delegate the decision to the AI. None of the subjects saw the AI’s decision before their delegation decision. Subjects were informed that if they chose to delegate, the AI’s decision would count as their own, and they would be compensated accordingly. They were explicitly told that they were delegating to an “artificial intelligence (AI).” Subjects received a base payment of $2.50 and an additional $0.05 per correctly classified image. They could also earn an extra $0.50 for completing a personality questionnaire at the end. This resulted in a total compensation ranging between $3.00 and $8.00.

The AI used was GoogLeNet Inception v3, a convolutional neural network trained on the ImageNet database containing 1,000 classes. The AI’s decision for each image was based on its highest certainty score, and the 10 class options shown to participants were those for which the AI reported the highest certainty. In our data set, the AI correctly classified 78 out of 100 images, achieving an accuracy of 78%. This is the aggregated AI accuracy across all images—that is, the share of images that the AI classified correctly. It reflects the AI’s actual realized performance over the entire image set.

After completing the classification task, subjects answered a survey on their perceptions of the AI, personality traits, and demographics. The survey included questions designed to serve as potential control variables, derived from the literature, which are included in the Online Appendix. These were self-assessed trust in the AI system, interpersonal trust, need for control, risk aversion, and the ability to override intuitive responses (cognitive reflection test).

4.2. Treatment Manipulations

Subjects were randomly assigned to one of four treatments. Before starting the main task, all subjects were informed that the AI had an aggregated accuracy of 78% for the given image data set. Beyond this general information on the AI’s performance, treatments varied in the type of AI system information provided. In the certainty conditions (treatments 2 and 4), subjects were shown the AI’s certainty score alongside the delegation option. This AI certainty is the AI’s self-estimated probability that its top prediction for the image is correct. We only displayed the AI certainty associated with the class that had the highest likelihood and was selected as the AI’s decision. Specifically, above the delegation choice, subjects saw a message stating, “The AI is x% certain about the class it chose,” along with their own previously stated certainty. They were informed that the AI’s certainty ranged from 0% (completely uncertain) to 100% (completely certain) and represented the AI’s calculated likelihood that its chosen class was correct. In the outcome conditions (treatments 3 and 4), subjects were shown whether the AI’s classification was correct after making a delegation decision, regardless of whether they actually delegated. After making their delegation choice, they were redirected to a new screen displaying the classified image, the class chosen by the AI, and a message indicating whether “The AI’s classification was CORRECT/INCORRECT.” To ensure that subjects understood the provided information of the AI, they had to pass an attention check tailored to their assigned treatment before beginning the main image classification task.

4.3. Participants

We conducted our study via CloudResearch, which accesses the participant pool of Amazon Mechanical Turk (MTurk) while allowing for the application of specific filters to ensure the quality of recruited participants. CloudResearch has been shown to recruit participants who provide meaningful responses and is considered one of the most reliable online data collection platforms available (Douglas et al. 2023). A total of 748 participants were recruited for the main study. Of these, 180 were automatically rejected by the system due to various reasons: 102 attempted to take the survey more than once, 48 did not meet the required screen resolution, and 30 failed the attention check. This resulted in 568 complete responses. Furthermore, we identified and excluded 8 responses suspected of being duplicates.² Therefore, our final sample consists of 560 subjects, with an average payment of $6.95. These data were first collected and reported in Taudien et al. (2022); here, we build on that work through an expanded and partly different set of analyses. Random assignment placed 137 subjects in Treatment 1, which serves as our control group (no certainty, no outcome); 144 to each of Treatments 2 (certainty, no outcome) and 3 (no certainty, outcome); and 135 subjects in Treatment 4 (certainty, outcome).

In terms of demographics, approximately 56% of participants identified as female, nearly 44% as male, and one participant preferred not to disclose their gender. Participant ages ranged from 18 to 74 years, with the largest share (32%) falling within the 35–44 age group.

4.4. Delegation and Performance Measures

In the following, we describe our outcome variables and provide an overview in Table 1. We operationalize our two main outcome variables at the task-instance level. Delegation frequency is measured as whether a subject delegated an image to the AI (1 = yes, 0 = no). To assess whether delegation improved performance relative to the subject performing the task themselves, we measure delegation effectiveness following prior work on effective system use (Burton-Jones et al. 2018). Delegation effectiveness indicates whether delegating a task resulted in improved performance (1 = yes) or not (0 = no).

Table 1. Overview of Outcome Measures

Additionally, we compare the treatments using performance measures aggregated at the subject level. First, we measure combined performance, which is the total number of correctly classified images after delegation decisions. This includes the subject’s own classifications for nondelegated images and the AI’s classifications for tasks they delegated. Second, we measure performance alone, which captures how many images participants classified correctly on their own, before seeing the option to delegate.

To better understand the reasons behind performance improvements, we distinguish between two types of delegation effects. The benefit of delegation quantity captures how much performance improves simply due to delegating more often. It is calculated as the difference between the expected performance under random delegation at the same delegation rate and performance alone. The benefit of delegation quality, on the other hand, captures how well subjects selected which tasks to delegate. It measures the difference between their realized combined performance and the expected performance under random delegation.

5. Results

First, we present the results of testing our preregistered hypotheses. Next, we provide exploratory analyses to further investigate the observed delegation behavior, followed by an analysis of the resulting performance.

5.1. Hypotheses Testing

In the following, we test our hypotheses regarding the average effects of our treatment manipulations on delegation frequency and effectiveness. Table 2 presents the results of our hypothesis testing. We begin by analyzing the effects on delegation frequency (Hypotheses 1(a), 2(a), and 3(a)), followed by an examination of delegation effectiveness (Hypotheses 1(b), 2(b), and 3(b)). In Table 2, Model (1), we estimate a mixed-effects logistic regression on delegation frequency (0 = no delegation, 1 = delegation). Given that each subject classified 100 images, we estimate mixed-effects models at the task-instance level, including random effects for subjects and images. Providing only AI outcome negatively impacts delegation frequency ($\beta$ = −0.397, p < 0.05), supporting Hypothesis 1(a). In contrast, providing AI certainty before a delegation decision significantly increases the odds of delegating ($\beta$ = 0.453, p < 0.05), contradicting Hypothesis 2(a). Additionally, showing both AI certainty and AI outcome leads to a significant positive interaction effect ($\beta$ = 0.589, p < 0.05), increasing the odds of delegation when AI certainty is combined with AI outcome information, thereby supporting Hypothesis 3(a).

Table 2. Hypotheses Testing

Next, we examine delegation effectiveness using a mixed-effects logistic regression model (Table 2, Model (3)). The dependent variable takes the value one if delegation resulted in improved performance compared with the user classifying the image themselves, and zero otherwise. Because the coefficient for AI outcome is negative and significant ($\beta$ = −0.423, p < 0.05), only seeing the AI’s outcome after a delegation decision decreases the odds that a subject delegates a task leading to performance improvement by 35% ($e^{-0.423}\approx$ 0.655). This finding supports Hypothesis 1(b). The coefficient for AI certainty is positive but not significant ($\beta$ = 0.240, p > 0.1), suggesting that only providing AI certainty before a delegation decision does not significantly affect delegation effectiveness. Thus, we do not find support for Hypothesis 2(b). Importantly, the interaction term between AI certainty and AI outcome is significant and positive ($\beta$ = 0.565, p < 0.05), supporting Hypothesis 3(b).

To ensure that our main findings are not confounded by individual differences in user confidence, we extend our analysis by including the level of user certainty as a control variable (Models (2) and (4)). The results indicate a strong negative effect of the level of user certainty on delegation frequency ($\beta$ = −8.371, p < 0.01) and delegation effectiveness ($\beta$ = −5.933, p < 0.01), suggesting that when being more confident, subjects are less likely to delegate in total and less likely to make a delegation decision that improves their performance. When controlling for user certainty, we find that the previously observed effects of our AI system information measures remain consistent: AI outcome on its own decreases delegation frequency ($\beta$ = −0.675, p < 0.01) and delegation effectiveness ($\beta$ = −0.538, p < 0.01). Contradicting our hypotheses but in line with our main results, providing AI certainty increases delegation frequency ($\beta$ = 0.729, p < 0.01) and has a positive effect on delegation effectiveness ($\beta$ = 0.448, p < 0.05). The interaction between AI certainty and AI outcome remains positive and significant for delegation frequency ($\beta$ = 0.605, p < 0.1) and delegation effectiveness ($\beta$ = 0.479, p < 0.1) when controlling for the level of user certainty.

5.2. Understanding Delegation Behavior

In this section, we further investigate the mechanisms underlying the mixed effects of AI certainty and AI outcome information on delegation behavior and what contributes to their positive interaction. To unpack why subjects behaved this way when exposed to different information about AI capabilities, we focus on delegation frequency. We first examine whether subjects strategically delegated tasks based on their perceived difficulty, distinguishing between easy and difficult images at the individual level (Table 3). We then analyze how users adjusted their delegation behavior after observing incorrect AI classifications (Table 4) and whether this adjustment depended on the level of the AI’s communicated certainty (Table 5). Finally, we explore how the absolute level of AI certainty influences delegation frequency across treatments (Table 6). Together, these analyses provide a more detailed picture of how AI system information shapes delegation frequency and under which conditions users’ decisions align with effective collaboration patterns.

Table 3. Comparison of Delegation Frequency Between Easy and Difficult Images

Table 4. Comparison of the Effect of an Incorrect AI Classification Among Treatments 3 and 4

Table 5. Analysis of a Misleading AI Certainty Signal

Table 6. Analyzing the Effect of the AI Certainty Level

Although AI certainty alone increased delegation frequency, it did not significantly improve delegation effectiveness. To explore factors contributing to effective delegation, we analyze whether subjects tended to delegate tasks that were difficult for them to classify, because these are the cases where AI support is most beneficial. To achieve complementary performance, users should delegate tasks they perceive as difficult rather than those they perceive as easy. We operationalize task difficulty at an individual level based on subjects’ self-reported certainty levels. We categorize images as difficult if the subject’s certainty was below their median certainty and as easy if their certainty was equal to or above their median certainty. We then conduct mixed-effects logistic regressions on delegation frequency separately for easy and difficult images (Table 3). For subjects who only saw AI certainty, delegation frequency significantly increased for easy images ($\beta$ = 1.847, p < 0.01), but not for difficult images ($\beta$ = 0.279, p > 0.1). In contrast, observing AI outcome alone had no significant effect for easy images, but significantly decreased delegation frequency for difficult images. This effect was mitigated by the positive significant interaction effect for difficult images, whereas no significant interaction effect was observed for easy images.

Thus, the positive interaction effect of combining AI outcome with AI certainty seems to be driven by difficult images. Seeing AI outcome alone led to a decrease in delegation frequency, but adding AI certainty could cancel this effect. A possible interpretation is that observing an AI making errors on difficult tasks reduces users’ willingness to delegate. However, when users are informed beforehand that the AI also perceives those tasks as difficult, this awareness helps mitigate the decline in AI usage.

To further test this interpretation, we analyze sequential effects of seeing different kinds of AI system information in the following. Specifically, we examine whether users reduced delegation for the next task directly after observing an incorrect AI classification. We only include those treatments that presented the AI’s outcome and estimate two models, one including subjects who did not see AI certainty (Treatment 3) and one including those who were provided with AI certainty (Treatment 4); see Table 4. We observe that subjects who only saw the AI outcome were less likely to delegate if the AI had incorrectly classified the previous image ($\beta$ = −0.227, p < 0.01). In contrast, for subjects who also saw AI certainty, the effect of a previous AI error on delegation frequency was not significant ($\beta$ = −0.073, p > 0.1). This suggests that although incorrect AI classifications typically reduce users’ willingness to delegate, this effect is mitigated when AI certainty is also provided.

If better mental models explain this mitigation effect, we should observe that users penalize AI errors (i.e., reduce delegation frequency) more in the case of a mismatch with provided AI certainty. Thus, we distinguish between cases where the AI certainty was high and cases where it was low. To categorize certainty in this way, we rely on the only explicit benchmark that all subjects received at the beginning of the experiment about the AI’s performance, which is its aggregated accuracy of 78%. Similarly to the previous analysis, we run two models considering subjects who (in addition to observing AI outcome) did or did not observe AI certainty (see Table 5). In Treatment 3, where AI certainty was not shown, subjects could not differentiate between these cases, and all incorrect AI decisions resulted in a significant decrease in delegation frequency for the subsequent image. In contrast, when AI certainty was provided (Treatment 4), only cases where AI communicated a high level of certainty for an incorrect decision led to a reduction in delegation frequency for the following image.

Although we hypothesized that showing certainty scores for an imperfect AI should lead to a reduction in delegation frequency, we observed the opposite effect. To analyze whether the specific level of AI certainty influences the delegation frequency, we extend our analysis by incorporating the level of AI certainty as a continuous variable (see Table 6). This approach allows us to examine how communicating different levels of AI certainty affects delegation frequency. We conduct separate regression analyses for subjects who did not see the AI outcome (Treatments 1 and 2) and those who did (Treatments 3 and 4). We find that when subjects did not observe the AI certainty score, higher AI certainty levels negatively impacted the likelihood of delegation. This effect was observed for subjects who did not see the AI outcome ($\beta$ = −1.793, p < 0.01) and for those who did ($\beta$ = −1.756, p < 0.01). This suggests that in the absence of AI certainty information, subjects were less likely to delegate cases where AI certainty would have been high, potentially because they felt confident making the decision themselves. When AI certainty was provided, displaying a low certainty level had a negative effect on delegation frequency without ($\beta$ = −3.262, p < 0.01) and with AI outcome ($\beta$ = −2.770, p < 0.01). However, at higher levels of AI certainty, the effect reversed, leading to increased delegation frequency. Specifically, when only AI certainty was shown, the effect of providing AI certainty on delegation frequency turns positive for certainty values above 67.1% (55.4% when AI outcome was provided as well).

Summarizing our findings, we conclude that the positive effect of presenting AI certainty alone seems to be driven by the (many) images with high levels of AI certainty. The positive interaction effect of adding AI certainty information to AI outcome can be related to better mental models, especially when seeing errors of images with low AI certainty.

5.3. Effect on Performance

In this section, we examine whether and how the observed delegation patterns translate into improved combined performance of humans and AI. We first compare performance across treatments to assess the overall impact of AI system information. We then decompose performance gains into benefits arising from delegation quantity and delegation quality to distinguish between increases driven by delegating more tasks and those driven by delegating the right tasks. Finally, we compare human and AI accuracy at the task level to contextualize the potential for complementary performance and quantify the realized improvement through informed delegation.

To assess these effects, we analyze the combined performance of humans and AI. Figure 3 presents the corresponding average performances across treatments. In Treatments 1, 2, and 4, average performance after delegation exceeds that of AI alone (78%). Notably, after a Bonferroni correction for the four treatment comparisons (adjusted $\alpha$ = 0.025), this increase is statistically significant only in Treatment 4, where AI certainty was displayed in combination with the AI’s outcome (t(134) = 3.72, p < 0.001). Here, combined performance also significantly exceeds subjects’ own performance without AI (paired t(134) = 11.01, p < 0.001; mean difference = 8.27). Treatment 4 is thus the only treatment in which subjects achieved complementary performance, surpassing both humans and AI individually, through delegation to AI.

These findings indicate that combining both types of AI system information enables users to make more informed delegation decisions, improving not only the frequency, but also the quality of delegation. To further investigate the sources of performance improvement, we distinguish between improvements driven by increased delegation frequency (i.e., how often users delegated) and those resulting from an improved understanding of the AI’s capabilities (i.e., how effectively users decided which tasks to delegate). To contextualize these improvements, we first compared the aggregated AI accuracy to that of human participants on a task-instance level. The AI outperformed the human (i.e., was correct, whereas the human was incorrect) in 17.2% of cases, whereas the human outperformed the AI in 12.1%. In the remaining 70.7% of task instances, both were either correct or incorrect. This distribution highlights a moderate performance advantage for the AI, but also reveals considerable complementarity between AI and human performance. It follows that simply delegating more tasks does not guarantee performance improvements because gains can only be realized if participants correctly identify the instances where the AI is more likely to succeed. To quantify this distinction, we compute the expected random delegation performance—that is, AI performance weighted by delegation frequency and human performance without AI weighted by one minus delegation frequency.

We define the benefit of delegation quantity as the difference between random delegation performance and performance alone, and the benefit of delegation quality as the difference between the actual achieved combined performance and random delegation performance. Table 7 presents the results of linear regressions comparing the performance across conditions, analyzing three key components: subjects’ individual performance without AI, the performance benefit of delegation quantity, and the performance benefit of delegation quality. The results show no significant differences between AI system information measures regarding individual performance or benefits derived from delegation frequency. However, for delegation quality, subjects achieved a greater benefit when they were provided with AI certainty alone and an even greater benefit when they received both AI certainty and AI outcome. Thus, the positive effect of providing both AI certainty and AI outcome seems to be driven by an increase in delegation quality rather than the mere increase in delegation quantity.

Table 7. Results from Ordinary Least Squares (OLS) Regression on Performance and Delegation Effects

6. Robustness Checks

In this section, we test the robustness of our findings through two additional experimental studies. The first robustness check examines how users behave when they have the option to leverage the efficiency advantages of delegating tasks to AI. Specifically, subjects are not required to classify each image themselves before deciding whether to delegate it to the AI. Instead, they can immediately delegate an image and proceed to the next one without classifying the image themselves. The second robustness check investigates whether informing users of the aggregated AI accuracy upfront influences the effects of displaying AI certainty and AI outcomes on delegation behavior.

6.1. Robustness Check 1: Including Delegation Efficiency

6.1.1. Purpose.

As defined previously, delegation involves transferring decision-making responsibility to the AI (Leana 1986, Baird and Maruping 2021). In our main study, subjects were required to classify each image themselves, before deciding whether they wanted to delegate the task to AI. Although this design allowed for within-subject analyses of delegation effectiveness, it did not allow for potential efficiency gains from delegation. In real-world settings, individuals often delegate tasks without attempting them first, using the time saved for other activities. To better replicate such scenarios, we designed this robustness check so that subjects only classified images themselves if they chose not to delegate them.

6.1.2. Study Design.

We implemented a between-subjects design with two treatments, using the same image classification task as in our main study with a reduced set of 50 images, among which the AI showed the same aggregated accuracy of 78%. In the baseline condition, subjects did not see information on AI certainty or outcome (equivalent to Treatment 1 of our main study). The treatment condition replicated Treatment 4 from our main study, where subjects saw both the AI certainty and AI outcome. In both conditions, subjects first viewed each image to be classified and decided whether to delegate the image to AI or classify it themselves. Only if they chose to classify the image themselves were they redirected to another screen where they could select 1 of 10 potential classes. If subjects delegated, the AI’s answer was taken as their classification, and they moved on to the next image. We recruited 199 subjects in January 2025 via CloudResearch, accessing the subject pool of Amazon MTurk. Random assignment placed 100 subjects in the baseline condition (no information) and 99 subjects in the treatment condition (information). All subjects received task instructions, had to pass a short quiz to ensure that they were human, and completed an exemplary classification to ensure that they understood the task. Subjects then classified 50 images presented in randomized order. Each subject received a base fee of $3.00 and an additional $0.05 for each correctly classified image. The average total payment was $5.08, and the experiment lasted an average of 29 minutes.

6.1.3. Results.

We analyze delegation frequency and combined performance in the following. Because subjects only classified images themselves when they chose not to delegate, we cannot compute delegation effectiveness because there is no counterfactual user classification when delegation occurs immediately. We find that subjects who saw both AI certainty and AI outcome delegated significantly more images to AI (M = 0.64, SD = 0.22) compared with subjects who did not (M = 0.42, SD = 0.31) (t = −5.719, $p<$ 0.01). Next, we study how displaying AI certainty and outcome affects the combined performance of subjects and AI in this delegation setting. Subjects in the baseline condition without AI information achieved an average combined performance of 78.6%, which is not significantly higher than the aggregated AI accuracy of 78% (t = 0.61, p = 0.54). Subjects in the treatment condition achieved a combined performance of 88%, significantly outperforming the AI alone (t = 14.486, $p<$ 0.01). To determine whether the increase in performance can be explained by an increase in delegation frequency, we set up two mixed-effects logistic regression models with random effects for subjects and images, using combined performance (0 = incorrect classification, 1 = correct classification) as the outcome variable. The results are presented in Table 8. We begin by analyzing the treatment effect in Model (1) and find that, consistent with our main study results, providing AI certainty and AI outcome information significantly increases performance. When we include a binary variable indicating whether subjects delegated an image (0 = no delegation, 1 = delegation), we find that the treatment effect remains consistent, whereas delegation itself shows a significant positive effect. Hence, our findings indicate that, consistent with our main study, presenting AI certainty and AI outcome improves performance by increasing the quality rather than the quantity of delegation.

Table 8. Effect of AI Certainty and AI Outcome in a Delegation Efficiency Setting

6.2. Robustness Check 2: The Role of Aggregated AI Accuracy

6.2.1. Purpose.

In our main study, all subjects were informed of the aggregated AI accuracy of 78%, ensuring that they entered the experimental task with a shared expectation of the overall performance of the AI system. This created a controlled environment for analyzing how AI system information on the task-instance level (AI certainty and AI outcome) affects delegation behavior. However, in real-world settings, the aggregated AI accuracy may not always be known to users or may be difficult to measure. In this robustness check, we therefore examine whether providing aggregated AI accuracy influences how users interpret AI system information on the task-instance level during their interaction with the system. Specifically, we examine how aggregated AI accuracy influences delegation behavior when no further AI system information is provided versus when subjects also receive information on AI certainty and AI outcome for each task.

6.2.2. Study Design.

We conducted a 2$\times$2 between-subjects experiment in which we manipulated whether subjects were shown the aggregated AI accuracy (no accuracy, accuracy) and whether they received additional task instance information, specifically ex ante AI certainty (the model’s confidence score) and ex post AI outcome (whether its prediction was correct) for each decision (no information, information). As in our main study, aggregated AI accuracy refers to the AI’s realized performance across the entire image set (78%). The experimental task and flow were identical to those in our main study, except that participants were required to classify a subset of 50 images. We recruited 408 subjects via CloudResearch, using the Amazon MTurk subject pool with specific quality filters, in January 2025. Random assignment placed subjects into one of four conditions: 102 subjects in the baseline condition (Treatment 1: no accuracy, no information), 100 subjects in the accuracy-only condition (Treatment 2: accuracy, no information), 105 subjects in the information-only condition (Treatment 3: no accuracy, information), and 101 subjects in the accuracy and information condition (Treatment 4: accuracy, information). Each subject received a base fee of $4.00, plus an additional $0.05 for each correctly classified image. On average, subjects earned $6.03 and completed the experiment in 49 minutes.

6.2.3. Results.

To analyze the effect of presenting aggregated and/or detailed AI information on our main outcome variables, we set up mixed-effects logistic regression models with random effects for subjects and images. The results are presented in Table 9. We find that displaying the aggregated AI accuracy upfront significantly increases subjects’ delegation frequency (Model (1)) and delegation effectiveness (Model (2)). In line with the results from our main study, the effects of detailed AI information (AI certainty and outcome) remain positive and highly significant, and we do not observe a significant interaction between providing aggregated and detailed information. Hence, the positive effect of providing AI certainty and outcome on delegation frequency and effectiveness holds even when subjects do not know the aggregated AI accuracy.

Table 9. Effect of Aggregated and Detailed AI Information

7. Discussion and Implications

7.1. Discussion of Results

Because users often fail to realize complementarities when delegating to AI, we examined how ex ante AI certainty and ex post AI outcome information shape delegation behavior. We analyzed two behavioral dimensions that influence the realization of complementary performance when humans delegate to AI, which are delegation frequency and delegation effectiveness. First, our results show that providing AI outcome information by itself negatively affects delegation frequency and effectiveness. Specifically, users reduce their delegation frequency after observing that AI is making errors. This aligns with findings on algorithm aversion, which suggests that users expect algorithms in general, and consequently also AI, to perform nearly perfectly (Dietvorst et al. 2015, Jussupow et al. 2024). As a result, they show low error tolerance for algorithmic systems. Interestingly, we find that aversion toward AI persists even when users are explicitly informed prior to collaboration about the aggregated AI accuracy (78%), which already signals that the system is imperfect. This persistence can be explained by research in cognitive psychology, suggesting that recently observed cues remain more memorable (Smith and Graesser 1981). Thus, observing errors for single decisions disproportionately influences users’ subsequent delegation decisions. Second, although we find that only presenting AI certainty increases delegation frequency, it does not lead to more effective delegation and, therefore, does not significantly improve combined performance. This does not fully support our hypotheses, in which we expected both delegation frequency and delegation effectiveness to decrease when certainty is provided by itself. Our exploratory analyses offer a more nuanced explanation. Although certainty alone does not increase delegation effectiveness, it does enable a modest improvement in delegation quality compared with random delegation, indicating that users extract some meaningful insights from the certainty scores. However, this improvement is insufficient to counteract the finding that users primarily delegate tasks that they find relatively easy rather than those where AI has a comparative advantage. Ideally, users should delegate tasks to AI that they find difficult, thereby leveraging complementary capabilities. Thus, AI certainty alone only encourages greater use of AI without enabling users to recognize complementary strengths. This highlights the critical importance of users forming accurate mental models regarding AI’s capabilities and simultaneously understanding their own limitations, known as metaknowledge (Jussupow et al. 2021, Fügener et al. 2022). Crucially, our findings demonstrate that combining AI certainty with AI outcome information significantly enhances both delegation frequency and effectiveness, ultimately enabling complementary performance. This improvement is driven not only by the increase in delegation frequency, but also by improvements in delegation quality, resulting from the synergy between the two pieces of AI system information. Specifically, we show that aversion toward AI is mitigated when users form realistic expectations of AI performance before delegation decisions. If AI signals a high probability of error in advance, subsequent errors do not significantly decrease the delegation frequency. Only signals where expectations are violated (indicating a high likelihood of correctness when the AI is, in fact, incorrect) still substantially reduce future delegation frequency. This pattern is consistent with expectation confirmation theory because users did not react to the error per se, but to the violation of their expectations (Brown et al. 2014). For our data set, AI certainty scores are strongly correlated with the actual accuracy of AI (Pearson’s r = 0.67, p < 0.01), allowing users to calibrate more realistic expectations, develop more accurate mental models, and improve delegation effectiveness. To further analyze this synergistic effect, we studied whether the aggregated AI accuracy of 78%, communicated to subjects at the beginning of the experiment, served as a reference point when interpreting the AI certainty at the image level. We chose the aggregated AI accuracy as a reference point because it provides users a clear, intuitive benchmark for contextualizing individual AI certainty values. This mechanism aligns with the anchoring heuristic, whereby salient reference points shape judgment under uncertainty (Tversky and Kahneman 1974). Indeed, our analyses confirm that users penalize AI errors more severely when these occurred despite the AI signaling a high level of certainty for a given image (above 78%), compared with lower AI certainty levels. Together, these findings suggest that AI certainty and AI outcome information individually serve as complementary reminders of AI fallibility. When combined, they help users more accurately infer AI error boundaries, thus improving their mental model of the system.

7.2. Theoretical Contributions

Our study provides several contributions to the literature on human-AI collaboration by deepening the theoretical understanding of how different forms of AI system information shape delegation behavior and how such information enables users to leverage human-AI complementarities. First, we extend research on the type of information that AI systems should provide by showing that the effects of AI certainty and AI outcome information cannot be understood in isolation, but emerge from how these signals jointly shape users’ expectations and mental models of AI performance. Previous work has typically studied these signals in isolation (Fügener et al. 2022, You et al. 2022, Liu et al. 2025) or in a way that made their effects difficult to disentangle (Zhang et al. 2020), producing mixed results. Moreover, prior studies examining AI certainty or AI outcome information have typically focused on outcomes such as willingness to use an AI system (Liu et al. 2025), general system use (Fügener et al. 2021), or trust in the system (Zhang et al. 2020, Rechkemmer and Yin 2022). Although informative, these measures do not capture whether users’ collaborative decisions were appropriate, leaving open the question of how such system information affects delegation effectiveness. We show that neither type of AI system information improves delegation effectiveness on its own, but that they interact positively. Drawing on expectation confirmation theory (Oliver 1980), we demonstrate why observing AI outcomes alone reduces delegation frequency: in the absence of accurate ex ante information about the AI’s likely performance, users interpret errors as expectation violations, which leads them to reduce delegation. Situation awareness theory (Endsley 1995) further explains why only the combination of ex ante and ex post information enables users to adapt their mental model to the specific context, supporting the comprehension and projection required for effective delegation. By identifying why isolated signals backfire and how their combination resolves these issues, we refine the theoretical understanding of how AI system information shapes delegation behavior and explain previously inconsistent findings (Zhang et al. 2020, Vaccaro et al. 2024). We thus recast AI certainty and AI outcome information not as independent signals, but as interdependent components whose joint provision is necessary to enable effective collaboration with AI.

Second, we advance research on human-AI complementarity by demonstrating that complementarity potential can be realized in delegation settings where users themselves allocate tasks and by identifying the behavioral conditions under which this occurs. Prior work has primarily focused on identifying the sources of complementarity between humans and AI, emphasizing differences in capabilities that give rise to performance advantages across tasks (Lu and Zhang 2025). Rather than explaining what enables these complementarities, we examine the conditions under which their potential can be realized in practice. Because delegation allocates each task to a single agent, our study focuses on the realization of inherent, between-task complementarity (Hemmer et al. 2025, Fügener et al. 2026). We explicitly assess whether the resulting combined performance exceeds that of both humans and AI individually; to our knowledge, we are the first to demonstrate complementary performance in a setting where humans themselves delegate to AI. Although prior work shows that such complementarities can, in principle, be leveraged (Fügener et al. 2026), it offers limited insight into the behavioral mechanisms through which they are realized when task allocation is determined by users rather than externally optimized or rule-based (Lai et al. 2022, Vaccaro et al. 2024). Our findings show that complementary performance does not arise from increased delegation alone. It is achieved only when users delegate appropriately, allocating tasks to AI in instances where it holds a comparative performance advantage. This identifies delegation effectiveness, rather than delegation frequency, as the behavioral dimension that distinguishes settings where complementarity potential is realized from those where it remains unrealized. In doing so, we complement prior work that has examined interventions to support human-AI complementarity in advice and integration settings (e.g., Bansal et al. 2021, Ma et al. 2023).

Third, we enrich the literature on algorithm aversion by identifying expectation calibration as a mechanism to mitigate aversion against AI. The foundational work by Dietvorst et al. (2015) introduced algorithm aversion in the context of rule-based prediction models, and subsequent studies have applied this concept to a wide range of information technologies, of which many were based on AI (Jussupow et al. 2024). Our study contributes to this evolving stream by identifying expectation calibration as a mechanism that shapes aversion in AI delegation contexts. Prior work shows that users reduce system use after observing algorithmic errors (Dietvorst et al. 2015). We replicate this effect in a setting with AI when users only observe AI outcomes and demonstrate that such reductions occur primarily after unexpected errors. When outcome information is paired with AI certainty, users only reduce delegation after unexpected high-certainty errors, not after expected low-certainty errors. Thus, we show that ex ante certainty information prevents reductions in system use when potential errors were signaled in advance. This extends existing work by clarifying the conditions under which aversion arises in AI contexts and complements post hoc mitigation approaches, such as allowing users to adjust algorithmic recommendations (Dietvorst et al. 2018).

Overall, we advance research on delegation to AI by conceptualizing delegation as a distinct form of system use and by introducing a theoretically grounded distinction between delegation frequency and delegation effectiveness. Prior work has conceptualized delegation primarily as a frequency decision (e.g., Liu et al. 2025). By building on the literature on effective system use (Burton-Jones and Grange 2013, Burton-Jones et al. 2018, Baird and Maruping 2021), we define delegation effectiveness as a performance-based measure of whether users assign tasks to AI in a way that leverages its comparative advantage. We show that delegation frequency and delegation effectiveness may respond differently to AI system information and are driven by different cognitive mechanisms. Consequently, increases in delegation frequency alone should not be interpreted as evidence of successful human-AI collaboration. By specifically studying delegation, we respond to a call for more IS research on delegation to AI (Baird and Maruping 2021). Delegation differs significantly from advice taking, because users transfer autonomy and responsibility for decision outcomes to AI, rather than retain it themselves (Baird and Maruping 2021). Whereas advice settings typically position AI as a supportive system (analogous to lower automation levels in autonomous vehicles), delegation involves higher levels of automation, with AI independently handling tasks. Therefore, having an adequate understanding of the AI’s error boundaries should be of even higher importance in delegation settings. Additionally, users lack ex ante knowledge of the AI’s decision in delegation settings. By contrast, users can adapt their mental model of the AI through the received recommendation (Bansal et al. 2019a) in advice settings. Thus, even if the user lacks an accurate mental model of the error boundaries of the AI system, they can still consider the AI’s recommendation before deciding whether to incorporate it into their final decision. In contrast, delegation requires users to form accurate expectations of AI capabilities before transferring decision-making responsibility, making the AI system information even more critical. Without a clear understanding of the AI’s capabilities, informed delegation decisions become challenging, which underscores the importance of providing meaningful AI system information.

7.3. Practical Implications

Our findings have several implications for the design and governance of AI systems in settings where users selectively delegate discrete tasks to an AI. A central challenge in such environments is that users make delegation decisions under information asymmetry, making it difficult to decide when the AI is more capable on a task than the human. Our results show that users make more effective delegation decisions through improved mental models of the AI’s error boundaries when systems communicate both ex ante certainty and ex post outcome information. Designing interfaces that provide both pieces of information can therefore strengthen users’ ability to allocate tasks to AI when it has a complementary advantage. Such features are particularly relevant in domains in which humans routinely decide whether to hand over individual tasks to an AI, such as content moderation, diagnostic prescreening, or fraud flagging. Our results caution practitioners against isolating AI system information. Providing ex post outcome information alone may reduce user delegation to AI, whereas combining it with ex ante AI certainty enables users to anticipate AI errors and adjust their expectations of the AI. The findings further speak to ongoing debates about automation boundaries and human oversight, as articulated in the European Union’s AI Act. The regulation emphasizes that, in high-risk domains, humans must remain meaningfully involved in oversight (Montag and Finck 2024). Our results suggest that well-designed delegation workflows in which humans selectively transfer responsibility for specific tasks based on AI system information may achieve strong collaborative performance without resorting to full automation. More broadly, our results indicate that organizations should view system transparency not as a static checklist of information to disclose, but as a dynamic design resource that shapes users’ mental models. Complementary system information can help users understand when AI is likely to outperform them and thereby support more effective human-AI collaboration. Organizations aiming to integrate AI as a teammate, rather than as a replacement for humans, should, therefore, focus on AI system information that helps users recognize complementarities and make informed delegation decisions.

7.4. Limitations and Future Research

Our study comes with some limitations that present opportunities for future research. Although AI certainty scores improve users’ mental models, their effectiveness relies on the certainty scores accurately reflecting the AI’s actual performance. Future research could explore whether users can learn to detect miscalibrated AI certainty when combined with outcome information. Additionally, providing outcome information for every decision may not always be feasible. Investigating whether users’ mental model calibration remains stable over time could provide valuable insights into how transparency mechanisms influence long-term decision making. We further acknowledge that the effectiveness of AI certainty may depend on both the level and variance of its certainty scores. Low variance may limit users’ ability to distinguish between tasks if they only see the AI’s certainty but no outcome, such that it would reduce the potential for learning from certainty information. When low variance implies that AI certainty is consistently low or consistently high, users are likely to adapt their delegation frequency accordingly and delegate less or more, respectively. Although such behavior would be appropriate if AI certainty accurately reflects performance, it limits the potential for realizing complementarities. In the case of persistently low certainty, the AI offers limited value beyond human judgment. Conversely, when AI certainty is consistently high, users may converge toward full reliance on the AI, reducing the opportunity for meaningful human contribution. By contrast, if certainty scores exhibit high variance, the effect depends on whether this variation meaningfully reflects performance differences. When the variance accurately reflects meaningful task-level asymmetries (i.e., there are some tasks where the AI outperforms humans and others where it does not), users can potentially identify these boundaries more easily, enabling more selective delegation and stronger complementarity. However, if the variance appears erratic or poorly aligned with performance, users may struggle to form reliable expectations, which can hinder learning and reduce delegation effectiveness. Future research could study whether these extreme cases of AI certainty impair the collaboration with humans.

Moreover, our study focuses on effective AI use by measuring improvements in combined performance. However, we do not examine efficiency gains, which can also be an important benefit of delegation (Baird and Maruping 2021). Effective delegation does not necessarily mean that delegation is efficient because users might benefit from delegating tasks because an AI can perform them more quickly or at lower cost. In addition, our focus is on how existing complementarities between humans and AI are realized through user behavior in delegation settings, rather than on characterizing the underlying capability differences that give rise to complementarity or on designing optimal integration mechanisms. Future research could build on this by examining how differences in human and AI performance across tasks affect users’ ability to identify when delegation is beneficial and, consequently, the realization of complementary performance. Although we consider our research to be generalizable to other descriptive AI applications, it may not be generalizable to algorithms in general. The employed image classifier represents a system that fundamentally differs from static, rule-based decision aids. Specifically, GoogLeNet Inception v3 is a convolutional neural network that has learned its own statistical representations from large-scale training data and generates probabilistic certainty scores for each decision. Such systems do not rely on human-defined decision rules but autonomously derive their own decision logic from data, enabling them to solve problems that humans cannot fully formalize, a phenomenon captured by Polanyi’s paradox (Polanyi 1966, Autor 2015). This capacity creates the basis for human-AI complementarity, because AI can reach solutions that differ from, and thus complement, human reasoning. Consequently, our manipulations and theoretical framing pertain to collaboration with an AI system rather than with any traditional algorithmic tool. Although analyzing potential differences in human behavior toward information on certainty and performance between AI and algorithms was not the focus of this study, we consider it to be an interesting angle for future research.

Finally, our study examines delegation for a generic task. Exploring whether delegation is affected differently by specific task characteristics could be a fruitful avenue for future research. Additionally, future research could analyze whether the beneficial effect of combining AI certainty before and AI outcome after an interaction extends to other forms of human-AI collaboration, such as advice settings.

7.5. Conclusion

Our research highlights the essential role of complementary system transparency in enabling effective human-AI collaboration. We show that providing accurate and timely information about an AI system through both ex ante certainty and ex post outcome information not only mitigates algorithm aversion, but also enhances users’ understanding of when to rely on AI. By helping users to form accurate mental models of AI error boundaries and calibrate their delegation decisions accordingly, providing combined AI system information fosters more effective and more strategically targeted use of AI. More broadly, our findings demonstrate that realizing the full potential of human-AI collaboration depends not only on more capable AI, but also on designing systems that actively support humans in recognizing complementarities between them. This calls for designing AI transparency that is aligned with users’ decision processes and facilitates more informed and well-calibrated collaboration.