Promotion Decisions and the Adoption of Explicit Potential Assessment

1. Introduction

Making promotion decisions is one of the most crucial, yet most challenging activities that firms face. Although promotion candidates are required to have the ability to perform well in a more complex future role, it is inherently difficult to accurately identify and assess the candidates’ potential. In this study, we examine the consequences of implementing a performance management system that adopted the explicit assessment of employees’ potential to facilitate promotion decisions.

In the organizational context, potential is typically used to describe an employee’s expected ability to advance and succeed in a higher position and reflects an employee’s readiness for the next job (Fernández-Aráoz et al. 2011). Here, it is essential to clearly distinguish between high potentials and high performers. High potential and high performance are not necessarily the same because skills required for the next level in the hierarchy are typically not perfectly correlated with skills needed to be the best performer in the current job, especially when moving toward leadership responsibilities (Baker et al. 1988, Grabner and Moers 2013, Benson et al. 2019). Hence, explicit assessment of potential becomes vital in identifying the best candidates for higher-level positions.

Firms recognize that the identification and retention of talented employees is among their most pressing concerns. With demographic shifts leading to waves of retiring employees and increased competition for a shrinking talent pool, the early identification of suitable promotion candidates at all hierarchical levels has become crucial for organizational success (Fernández-Aráoz 2014, Harris and Schwartz 2020). In response, firms are increasingly relying on potential assessment programs in practice (Strack et al. 2010, Fernández-Aráoz et al. 2011, Mercer and Mettl 2020, Deller 2023). However, only around 20% of organizations express confidence in their general ability to fill their potential pipeline and estimate a 40%–48% failure rate in identifying high potentials (Martin and Schmidt 2010, Fernández-Aráoz 2014, Talent Strategy Group 2015). This observation triggers the question if and how the explicit assessment of potential genuinely improves firms’ promotion decisions, an issue that thus far has largely been neglected in both the accounting and economics literature.¹ Although the requirement of providing an explicit potential rating is expected to shift the focus toward employees’ expected ability for the next job, a concern is that these evaluations might be uninformative (Cappelli and Keller 2014), which would potentially deteriorate promotion decisions. It is therefore an empirical question whether the benefits of a potential assessment system indeed materialize and, as a result, improve promotion decisions.

To shed light on this question, we examine the consequences of implementing a performance measurement system that transitioned from only evaluating performance to also providing an explicit assessment of potential. We conduct our study using proprietary performance evaluation data from an international professional service firm. The data cover the time period from 2010 to 2015, with the new system being implemented in 2013. In the old performance measurement system, employees received one overall rating for their current performance, whereas in the new system, they receive a rating for current performance and an additional potential rating, which combined results in a so-called nine-box model.²

We start our investigation by analyzing the association between the system change and the performance of promoted employees in their new job. In particular, using a set of complementary empirical specifications, we examine the preperiod versus postperiod difference in performance that follows after receiving a promotion, as the performance of promoted employees in the first year of their new job provides information about the quality of the underlying promotion decisions (Grabner and Moers 2013, Benson et al. 2019). Surprisingly, all of our tests consistently reveal that employees promoted under the new system show lower performance upon promotion compared with their counterparts promoted under the old system. This suggests that the potential assessment system is less effective in identifying candidates suitable for promotion to roles in a different, more complex task environment. This is particularly problematic given that the added value of a potential assessment system lies in identifying employees capable of performing tasks that are not yet part of their current job. Consequently, in a second step, we provide a more detailed analysis of the potential assessment system to uncover potential explanations for why the quality of promotion decisions might have deteriorated under this system.

As the explicit purpose of the potential assessment system is to guide promotion decisions, we first confirm that the potential ratings are indeed taken into account in promotion decisions. Our results indicate that the potential rating is a significant predictor of promotions in the postperiod and economically more relevant than the performance rating. Importantly, we also demonstrate that the new promotion rule indeed differs from the old promotion rule in terms of the type of people who are identified as promotion candidates as well as those who (do not) receive a promotion.

Given that the potential rating is incorporated into promotion decisions as intended, we continue by investigating whether the informativeness of the potential measure justifies its incorporation in the promotion decision. We analyze to what extent the potential rating is predictive of future performance for different groups of employees: promoted and nonpromoted employees (Grabner and Moers 2013). As the system is specifically targeted at assessing the ability to perform well at a new job in a different task environment, the potential rating should predict future performance in a new job (promoted) but less so, if at all, in the same job (nonpromoted). However, contrary to expectations, the results show that the potential rating predicts future performance for nonpromoted employees, but not for promoted ones. Combined with the previous analysis, these findings suggest that, although the firm makes promotion decisions in line with the outcome of the potential assessment, the information content of the potential ratings does, on average, not capture relevant information to identify suitable promotion candidates, potentially leading to an erroneous assessment of an employee’s ability to perform effectively in a different task environment.

We proceed with our analysis by exploring potential explanations for these results. Given that the evaluation of an employee’s future potential typically relies on the subjective assessment by supervisors, we identify conditions under which the evaluation quality of supervisors is higher versus lower, that is, the informativeness of potential ratings is higher versus lower. In a recent study using the same research site, Grabner et al. (2020) show that the firm rewards supervisors who exhibit greater rating differentiation among their subordinates. They argue that the firm views greater rating differentiation as reflecting higher evaluation quality because supervisors who show greater rating differentiation are expected to have provided more informative assessments. Consistent with their argument, we analytically show that a greater preference for accurate assessments leads to more time spent on gathering evaluation-relevant information. This not only leads to more informative assessments but also greater rating differentiation among employees. Using the same analytical model, we further show that a greater span of control puts a strain on the supervisor’s time, which leads to less informative assessments (Manthei and Sliwka 2019). As a result, we expect variation in evaluation quality based on two observable supervisor characteristics: (1) the extent to which the supervisor differentiates among employees when providing ratings (positively associated with evaluation quality (Grabner et al. 2020)) and (2) the supervisor’s span of control (negatively associated with evaluation quality (Manthei and Sliwka 2019)). Although we find no empirical evidence for span of control, we indeed observe that the potential ratings of differentiating supervisors are informative about future performance in the new job, whereas this is not the case for their less-differentiating counterparts. More importantly, we find that the earlier examined postperiod performance decline is significantly greater for promoted employees of supervisors who show low rating differentiation. Combined, our results lead us to conclude that the difficulty of evaluating employee potential reduces performance upon promotion because of inaccurate recommendations of supervisors who do not sufficiently differentiate in their ratings. These theory-consistent, cross-sectional results are particularly relevant as they highlight that our findings are unlikely to be firm-specific and apply more generally to firms moving toward the adoption of explicit potential assessments.

Our study contributes to the literature on performance measurement and promotions in several related ways. First, we apply the concept of predictive validity as a key design characteristic for performance measurement systems to a promotion setting. We show that the explicit assessment of potential is not necessarily useful in predicting future job performance in a new task environment. Given that such potential assessments nevertheless are, as intended, weighted in promotion decisions, our findings provide a novel explanation for the occurrence of the Peter Principle. Chan (2018) and Benson et al. (2019) show that the Peter Principle can occur by an overreliance on measures of current performance in promotion decisions because this provides, for example, incentives to perform in the current job. Complementing this evidence, our findings indicate that the Peter Principle might also be the result of relying on what appears to be a relevant measure, whereas this measure actually lacks the information content it is supposed to possess. We further identify variation in supervisors’ evaluation quality as a significant factor contributing to the lack of informativeness and, consequently, to the Peter Principle.

Second, recent studies show that supervisors differ in the extent to which they differentiate among subordinates’ performance levels, and that rating differentiation is positively associated with supervisors’ career prospects (Grabner et al. 2020) and subordinates’ motivation (Kampkötter and Sliwka 2018). The underlying assumption of these studies is that greater rating differentiation is associated with more informative assessments. We complement the literature by providing direct empirical evidence of this association. In particular, we show that potential assessments of supervisors who show greater rating differentiation are more informative than those of less-differentiating supervisors.

Finally, our results reveal that the explicit assessment of potential and its incorporation in promotion decisions is more challenging than firms might have anticipated. Our findings inform practitioners that implementing an explicit potential assessment system most likely requires implementing other, complementary control practices. In particular, such practices should focus on the supervisors and how to support them in executing the evaluation process to achieve desired outcomes.

2. Theoretical Background

2.1. Background and Related Literature

Research on performance measurement, and the choice of performance measures in particular, has a long tradition in the accounting literature. Numerous studies have examined which performance measures are used for incentivizing managers and how multiple measures are weighted to provide these incentives (Lambert and Larcker 1987, Bushman et al. 1996, Ittner et al. 1997). All these studies are rooted in the seminal work by Holmström (1979), who shows that any (costless) performance measure that provides incremental information about the agent’s effort should be incorporated into the provision of incentives. However, in the context of using performance measures for promotion decisions, information about effort is not necessarily relevant. An important role of promotions is to sort employees into the right job in accordance with their abilities (Baker et al. 1988, Gibbs 1995).

When assessing promotion candidates with the purpose of sorting, management seeks information about the extent to which an employee’s ability is relevant for the new job and task environment that is entered upon promotion. If the new task environment resembles the current task environment, management can conveniently rely on current task performance measures to predict an employee’s suitability for the new job. Yet, if the new job significantly differs from the current job, current job performance is not necessarily predictive of performance in the next job and management most likely has to resort to subjective assessments of promotability. For example, in their setting, Grabner and Moers (2013) are empirically able to disentangle an employee’s overall annual assessment into a component that reflects current job performance and a component that reflects a subjective assessment of expected ability (i.e., potential). They demonstrate that the greater the difference in jobs between hierarchical levels, the less current job performance is predictive of future performance in the next job compared with the current job, whereas the opposite holds for a subjective assessment of expected ability. They further illustrate that the weights assigned to these two performance dimensions in promotion decisions are consistent with the measures’ predictive validity. In essence, the informativeness of a performance measure for sorting purposes is reflected by the measures’ predictive validity regarding future performance in the next job.

However, neither assessing the potential to perform well in the next job nor relying on such assessments in promotion decisions is an easy task. Regarding the latter, Benson et al. (2019) examine whether firms promote sales workers who excel in their current job to the position of sales manager. Their empirical evidence indicates that, although past sales performance as a sales worker positively affects the likelihood of being promoted to sales manager, past sales performance is actually negatively associated with performance as a sales manager. In addition, they reveal that the extent to which a sales worker has experience collaborating with other workers on sales transactions is positively associated with subsequent performance as a sales manager, but this measure is ignored in the promotion decision. Their empirical findings are hence consistent with the Peter Principle, the costs of which can be substantial.³

Nevertheless, there are several reasons why firms might be willing to accept such costs and rely on performance in the current job in promotion decisions, irrespective of the measure’s predictive validity. First, the benefits of providing incentives in the current job might simply outweigh the costs of “ignoring” the sorting role of promotions (Grabner and Moers 2013, Benson et al. 2019). This can, for example, be attributed to the importance of tournament incentives (Lazear and Rosen 1981) or wanting to avoid workers shirking in their current job (DeVaro and Gürtler 2016). Second, performance in the current job can typically be measured more objectively, which can further improve incentives (Prendergast 1998, Manthei and Sliwka 2019). Finally, fairness concerns can push for a focus on current job performance because relative performance in the current job is more salient to employees and can influence their expectations regarding their promotion prospects (Chan 2018).

In addition to not incorporating informative measures in promotion decisions, the information content of the measures of potential to succeed in the next job can itself be problematic. In the context of nonfinancial performance measures, Ittner and Larcker (2003) argue that the difficulty in linking these measures to future financial performance is primarily driven by incorrect measurement. These problems are expected to be even more pronounced when it pertains to assessing an employee’s potential to perform well in the next job. Not only is it challenging to predict how potential will play out in the future, but defining potential and finding appropriate criteria are equally difficult (Mercer and Mettl 2020). Furthermore, because of its forward-looking character, management typically has to resort to subjective assessments when evaluating potential. However, subjectivity allows measurement error to enter the process of assessing potential, with supervisors being a major source of such error (Bol 2011, Berger et al. 2013, Kampkötter and Sliwka 2018). For example, Bol et al. (2018) show, in an auditing setting where tacit knowledge is assumed to signal managerial potential, that only supervisors who themselves have higher tacit knowledge recognize a subordinate’s tacit knowledge as relevant for identifying their potential to be a manager. As a result, the necessity to use subjectivity can undermine the information content of potential evaluations, and emphasizing such an assessment in promotion decisions can thus create inefficient job assignments.

2.2. Research Questions

Potential identification programs are typically implemented to guide promotion decisions with the purpose of sorting, with the ultimate goal to secure succession of leadership roles. In the absence of a potential identification program, employees typically receive one overall performance rating, which next to their performance in the current job might or might not include some implicit potential assessment. The prevalent approach in a potential identification program, however, is to explicitly separate the measurement of performance in the current job from the potential to perform in the next job (Lucy 2016). A popular talent management tool that is widely used by organizations to separate performance from potential is the nine-box model (Ruddy and Anand 2010, Cappelli and Keller 2014, Talent Management Institute 2021, Deller 2023). The model uses a grid with nine boxes to plot employees’ performance against their potential, allowing organizations to identify high-potential employees and plan their development and succession (see Appendix A). The usefulness of models like the nine-box model is not limited to any particular organization or industry (Indeed 2023), although it seems to be more commonly used by larger organizations with a large employee base (Ruddy and Anand 2010).

The main claimed benefit of the nine-box model is that it helps organizations identify employees who have the potential to take on leadership roles in the future and are up for promotion, allowing the firm to focus on their development and retention (Deller 2023, Indeed 2023). In this regard, Church et al. (2015) state that the primary reason for the design of the popular nine-box model is to avoid the occurrence of the Peter Principle, by focusing on more than just current performance. The nine-box model furthermore provides a structured approach to talent management, making it easier for organizations to assess their employees and plan for their development and support succession planning (Silzer and Church 2010).

A limitation of the nine-box model is that it still only focuses on employees’ performance and potential, ignoring other factors such as cultural fit, personality, and team dynamics, which can also impact employees’ success in a role (Silzer and Church 2010). However, the main limitation of the nine-box model that the practitioner-oriented literature highlights is the difficulty of assessing potential, and its distinction from current performance. The assessment heavily relies on subjective evaluations, which can result in employees being rated inaccurately on potential (Talent Management Institute 2021, Whatfix 2022). Cappelli and Keller (2014) even argue that there is little reason to believe that such assessments produce valid information.

Although explicitly separating performance in the current job from the potential to perform in the next job enjoys practical relevance and is popularized by the widespread use of the nine-box model, empirical evidence on the effectiveness of such an explicit split in measurement is absent thus far (Cappelli and Keller 2014). As a result, we first examine to what extent the explicit assessment of employee potential serves the purpose of improving promotion decisions by exploiting the change in a performance measurement system.

Research Question 1. To what extent does the explicit assessment of employee potential serve the purpose of improving promotion decisions?

Although it is an open question whether the explicit assessment of employee potential improves promotion decisions, on average, the main concern with potential identification programs highlighted by practitioners is the information content of the potential assessments. The academic literature on subjectivity suggests that there might be variation in informativeness among supervisors (Bol et al. 2018, Kampkötter and Sliwka 2018, Grabner et al. 2020). If so, then such variation likely translates into cross-sectional variation in improvements in promotion decisions. We therefore pose the following second research question.

Research Question 2. To what extent does the informativeness of explicit potential assessments vary among supervisors and can such variation explain cross-sectional variation in improvements in promotion decisions?

3. Research Setting

Our research site is an international professional service firm specializing in the fields of assurance, tax, strategy consulting, and corporate finance, representing four service lines the company operates in.⁴ The firm’s corporate hierarchy is organized into six hierarchical levels (plus top management). Although the first two ranks refer to nonmanagement employees, the promotion to the manager rank is typically accompanied by the acquisition of management and supervisor responsibilities.

The firm has implemented a structured career development and supervision program for employees to advance within the firm. It requires that each newly hired employee is permanently assigned to a supervisor who is an experienced professional and supports the employees in their development process. The program further defines specific performance goals and mandatory training sessions tailored to each hierarchical rank. On top of the mandatory aspects, employees are encouraged to establish individual goals in consultation with their supervisor. At the beginning of the career, the employees’ development focuses on technical skills, as well as team integration. Along with the advancement in the hierarchy, the area of responsibility enlarges, which extends the required skill set. Supervisory tasks include leading projects and assigning tasks to team members, but also the individual support of subordinates. At even higher levels, client communication and acquisition, as well as the strategic development of human capital become important.

3.1. Evaluation Criteria for Performance and Promotions Under the Old and New System

In order to evaluate employees and identify promotion candidates, the firm has an extensive annual review process in place. The annual performance evaluation process starts with the subordinates who provide a self-assessment to their supervisor.⁵ After a revision, the supervisors gather additional performance-related information about their subordinates, including project ratings and informal feedback from project leaders. During the individual performance review meeting with the subordinate, the subordinate’s performance is discussed based on the self-assessment and gathered information, as well as the achievement of the assigned and self-established goals, and further personal and professional development during the year. Subsequently, the supervisor subsumes all information about the subordinate’s capabilities into a supervisor rating.

After the completion of the individual meetings with the subordinates, all supervisors and top management (typically per office) gather to finalize the performance evaluation process in the calibration committee meeting.⁶ Based on the performance evaluation of the supervisor, the calibration committee discusses the performance of the subordinate. After deliberation, the calibration committee agrees on a final rating for each subordinate that reflects the subordinates’ performance relative to their peers. The final rating has direct compensation consequences in terms of bonus allocation and to some extent salary increases.

To determine the final rating for current job performance, the organization has established several criteria. One of the main criteria is the project rating that the employee receives after finishing a project.⁷ Further, the achievement of goals and objectives in accordance with the individual goal plan is considered during the assessment. Further performance indicators such as effective utilization, sales, engagement in special projects like recruiting activities, as well as integration into the team, and commitment to the organization are taken into account. In addition, the supervisor can access additional information such as multisource feedback from other colleagues. It is explicitly stated that the final rating is a reflection of current performance. These criteria are applicable in both systems.

Subsequent to the determination of the final ratings, the promotion candidates at each rank are compared and selected based on their current performance (final rating), but also with regard to competencies that are relevant in the next position. A promotion implies a jump to the next hierarchical rank, which includes a substantial change in task environment for all ranks except for the lowest (assistants). It also implies a significant structural increase in salary. Therefore, the accurate identification of potential for promotion candidates is crucial to be able to assess the fit between a candidate’s abilities and the demands of the next job. The criteria to determine potential, and hence to promote an employee, differ before versus after 2013 (pre- versus postperiod). Before 2013, guidelines proposed criteria such as “proven entrepreneurial and leadership skills” or “outstanding technical expertise,” which typically the supervisor assesses and shares with the calibration committee. Although these criteria are discussed, they are not formally documented in terms of a concrete rating under the old performance management system. As of 2013, the assessment, and hence identification, of potential has changed with the implementation of the nine-box model. The reason for implementation follows the argument that the battle for talent has become increasingly acute among firms that require highly skilled and well-educated people. The firm’s implementation decision is thus in line with a common trend among professional service firms to implement potential identification programs. The nine-box model was intended to allow the firm to more systematically and strategically manage its workforce. It should help leaders identify talent, as it marks out individuals who have the potential and performance to move up within the organization to take up senior roles.

Part of the nine-box model is the explicit assessment of potential, which is applied to all ranks with the exception of the lowest rank of assistants.⁸ On the horizontal axis, current performance is documented, with the left box including employees who received a final rating of one or two, the middle box for employees with a three or four, and the right box covering employees with a four or five.⁹ On the vertical axis, the employee’s potential is assessed. Low, medium, and high potential reflects whether the employee is well placed in the current role, has vertical potential for a broader and more complex role, or has vertical potential for a significantly broader and more complex role, respectively. To support the calibration committee in identifying the respective potential level, the respective supervisor fills out a questionnaire assessing three dimensions (aspiration, engagement, and ability) before the meeting, which results in an initial potential level to be discussed in the calibration meeting.¹⁰ Together with the final rating, the calibration committee then determines an employee’s position in the nine-box model, which enables them to make a final promotion decision. Thus, although the final decision on performance and potential ratings are made by the entire calibration committee (including the supervisor), the supervisor’s input typically is decisive.

It is important to note that both the performance rating and the potential rating are subjective in nature. For the performance rating, it is credible to assume that it captures current job performance. In determining the current performance rating, evaluators can rely on a number of signals such as subjective evaluations regarding project performance complemented with objectively measured dimensions such as effective utilization and overtime. In addition, in our setting, a calibration committee is in place that is adept at evaluating current performance and teasing out components that do not belong in this rating. We use the calibrated performance rating as measure of current job performance, which is also the key dependent variable in our analysis. Potential assessments, and by implication the prediction how well an employee will perform in a new role, are inherently more subjective as they involve predicting performance in a job with new, not-yet performed responsibilities. It is therefore unclear whether potential ratings indeed capture an employee’s suitability for an advanced role, which is a crucial part of the research questions we address.

3.2. Sample and Descriptive Statistics

We have data for a sample spanning a period from 2010 to 2015, capturing three years before the implementation of the new potential assessment system and three years afterward. The main sample consists of 7,709 employee-year observations pertaining to 3,618 unique employees. These employee-year observations relate to seniors (4,284), managers (2,098), and senior managers (1,327), which are the ranks exposed to the new potential assessment system, and it is this group that we focus on in the analyses and refer to as employees in the remainder of the paper, unless explicitly stated otherwise. Figure 1 provides a visual representation of the firm’s hierarchy and sample. When examining prepost differences, we (i) exclude 2012 as the transition year and (ii) focus on future performance, which reduces the sample to 4,429 employee-year observations, pertaining to 2,526 unique employees. Of these employee-year observations, 962 relate to promotions.

Descriptive statistics are provided in Table 1, differentiating between the preperiod (Panel A) and the postperiod (Panel B). The descriptives are economically fairly similar between the two periods, although there are some statistical differences. In the preperiod, the average performance rating is 3.63 and decreases to 3.52 in the postperiod (p < 0.01 for difference; two-tailed). The overall frequency of promotion decisions has also slightly changed, decreasing from 22% in the preperiod to 20% in the postperiod (p < 0.05 for difference; two-tailed), although there are no significant prepost differences within service line and rank.¹¹ The average age is 34.04 in the preperiod and 34.83 in the postperiod (p < 0.01 for difference; two-tailed). The percentage of males and average job tenure does not differ between periods, being just >60% and 2.8 years, respectively. Further, regarding the nine-box evaluations in the postperiod, the average potential rating equals 2.18, with 12.0% of the employee-year observations in the low potential category, 58.2% in the medium potential category, and 29.8% in the high potential category.

Table 1. Descriptive Statistics

The examination of the correlation table (Table 2) does not give rise to multicollinearity concerns. Of particular interest, however, is the strong correlation between the performance rating and potential rating in the postperiod, which equals 0.50 (p < 0.01; two-tailed). There are a number of potential explanations for this correlation, two of which are the following. Some part of current performance predicts how well an employee is expected to perform upon promotion because careers are designed vertically and some skills used in the current job will remain relevant in the new job. Alternatively, potentially ratings are, for better or worse, affected by the assessment of performance. Irrespective of these explanations, the correlation also implies that 75% of the variation in potential ratings cannot be explained by performance ratings (and vice versa). Whether this implies that these ratings are therefore also informative about performance upon promotion is one of the questions that we address.

Table 2. Correlation Matrix

4. Analysis of the Explicit Assessment of Employee Potential (Research Question 1)

To address our first research question, our main empirical analysis proceeds in two steps. In the first step, we analyze the association between the system change and how well promoted employees perform in their new job. In particular, we examine the prepost difference in performance upon promotion. In the second step, we provide a detailed analysis of the new system, in terms of the incorporation of the potential assessments in promotion decisions and the average informativeness of these assessments.

4.1. Association Between System Change and Performance upon Promotion

To examine the consequences of implementing a performance management system that adopts the explicit assessment of potential, we analyze whether employees’ performance after being promoted differs between the preperiod and the postperiod. That is, when employees get promoted to a new job, we examine whether performance in the new job differs between the preperiod and the postperiod. As a result, we specify the following regression model:

where i relates to the employee, and t to the year.

The dependent variable is future performance (PERFORMANCE_RATING_it+1), which relates to the final performance rating of the calibration committee in the year after the change in position, that is, performance in the new job.¹² The treatment variable is POSTPERIOD, which equals one after the implementation of the new system and zero before. It indicates whether there is a difference in future performance following a promotion decision in the pre- versus the postperiod. We include the current final rating (PERFORMANCE_RATING) to control for persistence in performance. We further control for the employee’s age (AGE), being male (MALE), and tenure in the current job (JOBTENURE) because prior research suggests that these are relevant predictors of rating outcomes (Gibbons and Waldman 1999, Lazear 2004, Correll et al. 2020). Finally, we include rank fixed effects and service line fixed effects.

The results are presented in column (1) of Table 3. The coefficient on POSTPERIOD is significantly negative (p < 0.01; two-tailed), showing that employees promoted under the new system have lower performance than those promoted under the old system.¹³ The performance decline after the introduction of the new system suggests that the promotion decisions that have been made using the new system in the postperiod have been less effective than in the preperiod, on average. We further find that future performance in the new job is higher based on (i) the higher performance in the current job (p < 0.01; two-tailed), (ii) the younger the employee (p < 0.05; two-tailed), (iii) the shorter the job tenure upon promotion (p < 0.01; two-tailed), and (iv) the employee is male (p < 0.05; two-tailed).

Table 3. Adoption of Explicit Potential Assessment and Performance upon Promotion: A Simple Pre-Post Comparison and Difference-in-Differences Analyses

Although the previous results are consistent with promotion decisions being less effective in the postperiod, there are several alternative explanations for the postperiod performance decline. In what follows, we sequentially discuss these alternative explanations and perform several tests attempting to rule them out. The first alternative explanation is that the coefficient on POSTPERIOD might capture other time-specific events or circumstances that impact performance. Although we are not aware of any other important changes that coincide with the introduction of the new system, the likelihood of such changes is not zero. Second, the new system might have induced a multitasking tradeoff wherein employees allocate effort away from current performance in favor of dimensions that relate more to potential (DeVaro and Gürtler 2016). Such a reallocation would be revealed by lower postperiod performance ratings. Third, the decline in performance ratings might not represent an actual decline in performance but rather the application of more stringent performance standards under the new system that includes potential ratings.

To rule out these three alternative explanations, we use nonpromoted employees as a control group. More specifically, we compare promotion observations to nonpromotion observations. If (any of) the three alternative explanations hold, then we should observe a similar performance decline for nonpromoted employees, because all three alternative explanations equally apply to this control group. We therefore estimate the following difference-in-differences equation:

where PROMOTED_it equals one if employee i receives a promotion in year t and zero otherwise. The results, presented in column (2) of Table 3, show a significant negative coefficient on the treatment variable POSTPERIOD × PROMOTED (p < 0.01; two-tailed). These results are robust to using a two-way fixed effects estimation (employee and year fixed effects) instead of controlling for the current performance rating, as well as two-way fixed effects estimation that includes controls for subdimensions of performance in the current job; in both cases, the coefficient on POSTPERIOD × PROMOTED is significantly negative (p < 0.05, two-tailed; see the Online Appendix, Table OA.A2). This indicates a greater decline in postperiod performance for promoted employees compared with nonpromoted employees, which is inconsistent with the first three alternative explanations driving the main results.

There are potentially two additional alternative explanations. First, the average performance decline might reflect employees’ postpromotion dips that are exacerbated by the new system emphasizing the importance of promotions in general. Relatedly, this heightened importance might have increased supervisors’ fairness concerns in the postperiod and triggered them to provide lower performance ratings for promoted employees in an attempt to minimize compensation differentials across employees. To rule out these two alternative explanations, we use promoted assistants as control group. If the two alternative explanations hold, we should observe a performance decline for promoted assistants that is similar to the treated group (promoted nonassistants). We estimate the following difference-in-differences equation for promotion observations:

Column (3) of Table 3 reveals a significant negative coefficient on the treatment variable POSTPERIOD × NONASSISTANTS (p < 0.05; two-tailed).¹⁴ That is, the decline in postperiod performance for promoted nonassistants is greater than for promoted assistants, which is inconsistent with the alternative explanations driving the main results.

Nevertheless, the assumption that the latter two alternative explanations equally apply to promoted assistants might not hold because these explanations could be less relevant at the first career step. To provide further evidence that lower-quality promotion decisions are more likely to explain the performance decline, we exploit the empirical strategy developed by Benson et al. (2019). Their strategy focuses on estimating the following equation:

The dependent variable equals performance upon promotion for those employee-year observations that involve a promotion and zero otherwise. As a result, the coefficient on PROMOTED, that is, ${\beta }_1$, is an estimate of average performance upon promotion (see Benson et al. (2019) for details). If Equation (4) is estimated using Ordinary Least Squares (OLS), then ${\beta }_1$ is the estimate of average performance upon promotion for all promoted employees. If Equation (4) is estimated using an instrumental variables approach with PROMOTED being instrumented, then ${\beta }_1$ is the estimate of average performance upon promotion for promoted employees that are compliers to the instrument. To understand how these two estimates can be informative about the quality of promotion decisions, assume that the instrument completely randomizes the promotion decision. If average performance upon promotion for all promoted employees (OLS estimate of ${\beta }_1$) is similar to that for compliers to the instrument (Instrumental Variable (IV) estimate of ${\beta }_1$), then the quality of the firm’s promotion decisions is not any better than making random promotion decisions. Thus, higher-quality promotion decisions are captured by a larger difference between the average performance upon promotion for all promoted employees and for compliers to the instrument, that is, a bigger difference between the OLS estimate and the IV estimate.¹⁵ Alternatively, when comparing the preperiod to the postperiod, a difference in OLS estimates between periods without a matching difference in IV estimates between periods reflects a change in the quality of promotion decisions.

We use this empirical strategy to estimate, for the preperiod and postperiod separately, the average performance upon promotion for (i) all promoted employees and (ii) promoted employees that are compliers to the instrument. In line with Benson et al. (2019), we use as instrument the average promotion rate per rank, per service line, per year, leaving out the focal employee and other employees at the same rank and service line that are supervised by the same supervisor. Compliers to the instrument are promoted because of a lucky draw of the instrument, irrespective of their performance or potential (“quasi-randomly promoted employees”). More specifically, compliers are employees who got promoted because average promotion rates for their group were high but would not have been promoted had it been slightly lower. In Online Appendix B, we discuss in detail the validity of our instrument and the identifying assumption that underlies our analysis based on the pattern in the four different estimates. Regarding the latter, the identifying assumption is that any alternative explanation that applies to promoted employees on average also applies to quasi-randomly promoted employees that come from this group of promoted employees.

Table 4 presents the OLS and IV estimates for the preperiod and postperiod. The first row of Table 4 shows that average performance upon promotion for all promoted employees is 0.20 lower in the postperiod compared with the preperiod (p < 0.01; two-tailed), which is similar to the results in column (1) of Table 3. More importantly, the second row shows that average performance upon promotion for quasi-randomly promoted employees is not different between periods (p = 0.95; two-tailed). The observed difference in OLS estimates without a matching difference in IV estimates is consistent with a decrease in the quality of promotion decisions.

Table 4. Adoption of Explicit Potential Assessment and Performance upon Promotion: All Promoted Employees vs. Quasi-Randomly Promoted Employees

Collectively, the results in Tables 3 and 4 suggest that promotion decisions are less effective in the postperiod when potential is explicitly assessed. However, it remains difficult to cleanly identify the average performance effect in the absence of objective measures of performance and completely rule out the alternative explanations.¹⁶ We therefore examine the new system in more detail to provide additional evidence.

4.2. Promotion Decisions Under the Old vs. New System

A change in the quality of promotion decisions after the new system introduction implies that the new promotion rule differs from the old promotion rule, resulting in a different composition of the (non)promoted workforce. To examine whether this is indeed the case, we first estimate these promotion rules by estimating the following probit regression models per period¹⁷:

where i relates to the employee, t to the year, Equation (5a) to the preperiod, and Equation (5b) to the postperiod.¹⁸ We include the square term of LN_JOB_TENURE because its association with promotion likelihood is very likely to be concave.¹⁹

The results in Table 5 indicate that, although the performance rating is a significant predictor of promotions in both periods, the coefficient (and marginal effect) is significantly smaller in the postperiod (p < 0.01; two-tailed). In addition, the potential rating is a significant predictor of promotions in the postperiod and economically more relevant in terms of marginal effect than the performance rating.²⁰ Although these results demonstrate, as could have been expected, that potential matters in promotion decisions in the postperiod to the detriment of performance ratings, this does not necessarily imply that the resulting promotion probabilities differ from those when using the old promotion rule. In essence, it may not necessarily lead to different promotion decisions.

Table 5. Role of Performance Ratings and Potential Ratings in Promotion Decisions

To provide more direct evidence of the difference in outcomes between the old and new promotion rule, we (i) predict the promotion probabilities in the postperiod using the old promotion rule by applying the coefficients from column (1) of Table 5 to the postperiod data, (ii) predict the promotion probabilities in the postperiod using the new promotion rule by applying the coefficients from column (2) of Table 5 to the postperiod data, and (iii) create quartiles within each rank, service line, and year for each set. This procedure allows us to compare the classification of employees, in terms of their expected probability of promotion, based on the old versus new promotion rule.

Panel A of Table 6 tabulates these two sets of quartiles. The results show that 74.7% are on-diagonal and 25.3% are off-diagonal, which implies that in approximately 25% of the cases (the size of a quartile), the two promotion rules classify employees in different quartiles of promotion likelihood. To examine whether and how these different likelihoods translate into differences in actual promotion rates, it is of particular interest to compare the employees in the upper right quadrant to the employees in the bottom left quadrant. That is, we compare the actual promotion rates for employees who have a low likelihood of receiving a promotion under the old rule (Q1_old and Q2_old) but a high likelihood under the new rule (Q3_new and Q4_new) versus employees who have a high likelihood of receiving a promotion under the old rule (Q3_old and Q4_old) but a low likelihood under the new rule (Q1_new and Q2_new). Panel B of Table 6 indicates that employees in the upper right quadrant are almost seven times more likely to be promoted than employees in the bottom left quadrant (17.07% versus 2.48%), which shows that the two promotion rules lead to significantly different outcomes. In addition, when comparing the composition of the nonpromoted workforce in the preperiod versus postperiod, we observe in Panel C of Table 6 that the distribution of employees across the quartiles of the old promotion rule is significantly different between periods (${\mathrm{\chi }}^2=7.78$; p = 0.05). In particular, relatively more Q4_old employees and fewer Q1_old–Q3_old employees are observed among the nonpromoted in the postperiod.²¹ Overall, our findings confirm that the new system that explicitly incorporates potential assessments leads to significantly different promotion decisions, resulting in a different composition of the (non)promoted workforce.²²

Table 6. Differences in Outcomes Between the Old and New Promotion Rule

4.3. How Informative Are the Potential Assessments?

Given that the potential ratings are incorporated into promotion decisions in the intended way, a possible reason for the performance decline in the postperiod is that these potential ratings actually do not capture the distinct potential to perform well in a (different) future job. Therefore, we proceed with an examination of the informativeness of the potential ratings. In line with Grabner and Moers (2013), we define informativeness as the predictive validity of the potential rating for future performance in a different job. In particular, we investigate the predictive validity of the potential rating for future performance in a different job (promotion group) and compare it to the predictive validity for future performance in the same job (nonpromotion group) using the following model²³:

where i relates to the employee, t to the year, and $X_{it}$ is a vector of control variables and fixed effects also used in Equation (2). The dependent variable equals future performance in the new job for those employee-years that involve a promotion, whereas it equals future performance in the same (current) job for those employee-years that do not involve a promotion. If the potential assessment is informative about an employee’s potential to perform in the next role, then (i) this should be reflected in the employee’s performance in the subsequent year when being in the new job, that is, ${\delta }_0+{\delta }_1>0$, and (ii) the predictive validity of the potential rating for future performance should be higher in the presence of a promotion compared with the absence, that is, ${\delta }_1>0$.²⁴

Table 7 presents the results on the predictive validity of the potential rating for future job performance for promoted and nonpromoted employee-years separately. The results first of all show that current performance predicts future performance in the next job less than when staying in the same job (p < 0.01; two-tailed), although significant in both cases. This is consistent with the idea that the knowledge and skills needed for the current job are different from, although related to, the knowledge and skills needed for the next job, which also substantiates why incorporating the concept of potential is relevant in the first place. However, we also find that the potential rating predicts future performance in the same job (p < 0.01; two-tailed), but not in the next job (p > 0.10; two-tailed), with the difference not being statistically significant. That is, ${\delta }_0+{\delta }_1\approx 0$ and ${\delta }_1\approx 0$.²⁵ The results thus show that the potential rating is informative of future performance in the same job but not of future performance in the new job, revealing that the potential rating is lacking the informativeness it is supposed to possess.

Table 7. Predictive Validity of Performance Ratings and Potential Ratings for Future Performance

Combined with the previous analyses, our results suggest that the weighting of the potential rating in promotion decisions is not consistent with its informativeness. That is, although the firm makes promotion decisions in line with the outcome of the potential rating, the information content of the potential rating does, on average, not capture the relevant information to identify the right promotion candidates, potentially leading to an incorrect assessment of how well an employee can perform in a different task environment. Paradoxically, this suggests that explicitly assessing potential can actually result in the occurrence of the Peter Principle rather than its avoidance.

5. Supervisor as a Source of the Peter Principle (Research Question 2)

We continue our analysis by investigating potential explanations for the finding that the potential rating, on average, lacks informativeness. By definition, evaluating an employee’s future potential relies on the subjective assessment of supervisors. Hence, our second research question explores the possibility that there is variation in evaluation quality and that some supervisors are better able and/or more willing to make potential assessments that are informative than others. To that end, we identify conditions under which the evaluation quality of supervisors is higher versus lower; that is, the informativeness of potential ratings is higher versus lower. In particular, we expect variation in evaluation quality based on observable supervisor characteristics.

Using an analytical model (see Online Appendix C), we derive two observable characteristics that are associated with evaluation quality: (1) the extent to which the supervisor differentiates among employees when providing ratings (positively associated with evaluation quality) and (2) the supervisor’s span of control (negatively associated with evaluation quality). Regarding the first, supervisors can have, in general, a preference for a more “outcome-based team culture” or for a more “harmonious team culture.” In terms of performance evaluation, the former relates to a preference for accuracy of ratings, whereas the latter relates to a preference for equity among the agents. In contrast to equity preferences, a stronger preference for accuracy triggers the supervisor to collect evaluation-relevant information about each subordinate, which results in both more informative assessments and greater rating differentiation among subordinates (Berger et al. 2013, Kampkötter and Sliwka 2018, Manthei and Sliwka 2019). Although this does not imply that supervisors with equity preferences who differentiate less are necessarily bad supervisors, the firm in our research setting penalizes low differentiators with reduced career opportunities (Grabner et al. 2020), signaling that differentiation is part of better performance evaluation practices. In line with Grabner et al. (2020), we therefore assume that greater rating differentiation reflects higher supervisor evaluation quality and expect that, despite its inherent difficulty, differentiating supervisors are more likely to provide more informative potential assessments of an employee than their less-differentiating counterparts.

Hypothesis 1.

All else equal, a supervisor’s rating differentiation is positively associated with the informativeness of potential assessments.

Next, following insights from Manthei and Sliwka (2019), we propose that variation in the span of control, that is, the number of employees who report to a particular supervisor, explains variation in the informativeness of potential assessments. As argued above, the informativeness of subjective assessments depends on the time allocated to this assessment and the span of control puts a strain on the supervisor’s time. We analytically show in Online Appendix C that when the number of employees that needs to be assessed increases, less time is spent on both performance and potential assessments, resulting in less informative ratings. Under the plausible assumption that supervisors care more about accurate assessments in the current job compared with future potential for the next job (see Online Appendix C for details), potential assessments become even less informative with a greater span of control, because the time spent is allocated away from these assessments in favor of performance assessments. As a result, we expect that with a greater span of control, the time allocated to assessing employees’ potential decreases, resulting in decreased informativeness of the potential ratings.

Hypothesis 2.

All else equal, a supervisor’s span of control is negatively associated with the informativeness of potential assessments.

To test these predictions, we examine the predictive validity of performance and potential ratings for future performance for groups split on (i) nonpromotion versus promotion observations and (ii) low versus high values on the supervisor variable of interest (HiXSV). These supervisor variables are (i) rating differentiation (low versus high, based on a median split of supervisors differentiation of performance ratings), (ii) past performance (low versus high, based on the performance rating of the supervisor being less than or equal to three versus higher than three), and (iii) span of control (low versus high, based on median split).²⁶ The first and third variables directly follow from the analytical model, whereas the second variable is used as an alternative measure of supervisor evaluation quality.²⁷

We adapt Equation (6) and estimate the following equation:

Panel A of Table 8 presents the results for low versus high rating differentiation by the supervisor. We find that, for low differentiators, the potential rating predicts future performance in the same job (p < 0.01; two-tailed) but not in the next job (p > 0.10; two-tailed), and the latter is significantly different from the former (p < 0.01; two-tailed). For high differentiators, we find that the potential rating predicts future performance in the same job (p < 0.01; two-tailed) as well as in in the next job (p < 0.10; two-tailed), with no significant difference. In addition, the potential rating of high differentiators is significantly more predictive of future performance in the next job than the potential rating of low differentiators (p < 0.05; two-tailed). Thus, the potential ratings of high differentiators are informative of future performance in the next job and significantly more so than the potential ratings of low differentiators.

Table 8. Supervisor Characteristics and the Predictive Validity of Performance Ratings and Potential Ratings for Future Performance

The results based on supervisor’s past performance, presented in Panel B of Table 8, show a pattern that is to some extent in line with the results for rating differentiation. That is, for low past performance of the supervisor, we find that the potential rating is neither predictive of future performance in the same job nor in the next job. In contrast, for high past performance, we find that the potential rating is predictive of future performance in both the same job and the next job (for both, p < 0.05; two-tailed). However, none of the coefficients are statistically different between low and high past performance of the supervisor.

Finally, regarding the supervisor’s span of control (Panel C of Table 8), we find that the results do not differ between the sample splits. That is, the potential rating is predictive of future performance in the same job but not the next job, irrespective of the supervisor’s span of control. A potential explanation for the lack of difference is that a greater span of control provides more benchmarks for each employee, which allows for a better evaluation. That is, if performance about employee A can be used to make assessments about employee B (and vice versa), then a greater span of control by itself provides more signals about each employee and thus, all else equal, leads to more informative assessments. This might counterbalance the negative time effect that comes with a larger span of control, overall leading to no significant differences.

In sum, the results in Table 8 reveal that it is the supervisor’s rating differentiation that significantly explains cross-sectional variation in the informativeness of potential ratings for performance upon promotion. Additional untabulated tests reveal that the firm, when making promotion decisions, does not differentiate between potential ratings coming from supervisors with low versus high rating differentiation.²⁸ These results suggest that, if the on average lack of informativeness of potential ratings drives the on average postperiod decline in performance upon promotion, then the supervisor evaluation quality, reflected by rating differentiation, also should explain cross-sectional variation in the postperiod performance decline. That is, the postperiod performance decline should be greater for supervisors who differentiate less compared with supervisors who differentiate more.

To examine whether rating differentiation explains cross-sectional variation in differences between preperiod and postperiod performance upon promotion, we re-estimate Equations (1)–(3) and allow the coefficients to vary between low and high rating differentiation.²⁹ The treatment variable of interest is the interaction between POSTPERIOD and LOW_DIFFERENTIATION, which is expected to be negative. We present the results for Equations (1)–(3) in columns (1)–(3) of Table 9, respectively. For Equation (1), which examines pre-post differences for promoted employees, we find that the postperiod performance decline is significantly greater for promoted employees of low differentiators compared with high differentiators (p < 0.05; two-tailed). When using nonpromoted employees as a control group, that is, Equation (2), we find a significantly greater performance decline for promoted employees compared with nonpromoted employees in case of a low differentiating supervisor compared with a high differentiating supervisor (p < 0.05; two-tailed). We find similar results when using promoted assistants as a control group, that is, Equation (3), although the difference between low and high differentiators is only one-tailed significant (p = 0.19; two-tailed).

Table 9. Pre- vs. Postperiod Performance upon Promotion Conditional on Supervisors’ Rating Differentiation

Overall, we find that the informativeness of explicit potential assessments systematically varies with supervisors’ rating differentiation, consistent with the theoretical prediction that supervisors who show greater differentiation have invested more time in assessing employees and therefore provide more informative assessments.³⁰ We also find that rating differentiation explains cross-sectional variation in the postperiod performance decline, with this decline being significantly greater for promoted employees of supervisors who show low rating differentiation. Combined, our results lead us to conclude that the difficulty of evaluating employee potential reduces performance upon promotion because of inaccurate recommendations of supervisors who do not sufficiently differentiate in their ratings. As a result, the explicit assessment of potential can lead to the occurrence rather than avoidance of the Peter Principle, and supervisors are a source of such occurrence by providing uninformative potential assessments.

6. Conclusion

In this study, we provide evidence on the consequences of implementing a potential measurement system. In the preperiod, the assessment information was limited to one overall rating that is supposed to capture current performance, and no separate information signal for potential was available. Thus, adding a separate measure for potential should help the firm in identifying suitable promotion candidates, ultimately leading to a better job assignment of employees.

However, our results paint a different picture and reveal an inconsistency in the weight placed on the potential rating in promotion decisions and the information content of these ratings. The firm seems to base promotion decisions on the outcome of the potential assessment process, assuming that these ratings correctly identify employees who have the potential to perform well in a different job. However, differences in potential ratings do not uniformly explain differences in future performance of promoted employees, implying that, on average, these ratings do not correctly capture the relevant information to identify the right promotion candidates. Our cross-sectional tests show that the predictive validity of the potential ratings is systematically related to the characteristics of the supervisors involved in these subjective assessments, and in particular to the supervisor’s degree of rating differentiation. That is, potential ratings of less-differentiating supervisors lack informativeness, thereby compromising the promotion decisions that the firm makes based on these assessments. Indeed, we find that performance upon promotion under the newly introduced potential assessment system is lower for subordinates who are guided by less-differentiating supervisors. Most importantly, these findings suggest that paradoxically, supervisors who are entrusted with supporting and guiding their subordinates’ career development paths can themselves be a critical source of the Peter Principle. Thus, careful consideration on who to trust with supervisor responsibilities is a prerequisite for the successful identification of promotion candidates, even if a uniform well-structured and thus very costly potential assessment process involving many information sources is in place.

Despite the observed differences among supervisors, the firm does not distinguish among potential ratings coming from different type of supervisors when making promotion decisions. This could be due to the firm being uninformed about the insufficient predictive validity of the potential ratings provided by a subset of the supervisors or these supervisors being able to convince the firm of the claimed accuracy of their potential assessments. But even when the firm is informed and can act on it, it is unlikely that doing so would improve the situation. If supervisors rationally anticipate that their potential ratings receive less weight in promotion decisions, they have even less of an incentive to invest in the informativeness of their potential assessments, undermining the system even further.

A natural follow-up question is then what the firm can do to mitigate the problem of uninformative potential assessments. In our analytical model, uninformative assessments are the result of supervisors’ conscious cost-benefit tradeoffs, where they tradeoff costly time investments with the benefits such investments provide. For example, supervisors who show a greater preference for equity have less to gain from more information and therefore limit their costly information acquisition, which results in less informative assessment. In this case, mitigating the problem of uninformative potential assessments requires the firm to directly change the benefits and costs. To do this, the firm can inform supervisors about the negative impact that inaccuracies have on firm decision making and further allow for sufficient supervision time. To the extent that uninformative assessments are also the result of unconscious errors, the firm can provide targeted training programs that make supervisors aware of the existence of such errors. Whether such mechanisms are effective in inducing informative potential assessments is an open empirical question. More generally, the lack of informativeness might also be the result of taking a “one size fits all” approach to potential assessment, relying on standardized questionnaires. Such an approach ignores the specifics of a particular setting, and therefore a more tailor-made approach might be needed to better characterize potential within a firm. To conclude, our results suggest that in order to be successful, an explicit potential assessment system cannot be implemented in isolation and needs to be accompanied by the introduction of other complementary control practices. This provides ample opportunity for future research on potential assessment.