How Much Do Industry, Corporation, and Business Matter, Really? A Meta-Analysis

1. Introduction

The academic field of strategy seeks to explain differences in firm performance (Rumelt et al. 1994, Nag et al. 2007). Consider, for example, Procter and Gamble’s business of manufacturing diapers for babies and Boeing’s business of building military aircraft for governments. Studies have found that three factors typically account for most of the performance differences between such businesses (Schmalensee 1985, Hansen and Wernerfelt 1989, Rumelt 1991, Roquebert et al. 1996, McGahan and Porter 1997): the industry in which the business operates (here, baby care versus defense industry); the corporate parent, if any, to which the business belongs (e.g., Procter & Gamble versus Boeing); and the business itself (i.e., the specifics of each business). Although consensus exists that industry, corporate, and business effects together explain most differences in performance, there is ongoing debate about how much each of these factors explains (Hough 2006, Misangyi et al. 2006, Bou and Satorra 2010, Karniouchina et al. 2013, Stavropoulos et al. 2015, Zavosh and Dibiaggio 2015).

I argue that achieving consensus requires consistent measures to quantify the contribution of industry, corporate, and business factors to firm performance. Studies have used three different effect size measures: sum of squares, variance, or standard deviation. These various measures yield different results for a given sample, which precludes the direct comparison of samples using different measures. Whereas the variance and standard deviation measures are easy to convert from one to the other, this is not the case for the sum-of-squares and variance measures—which are the measures used most often.

I employ simulation analysis to show that the sum-of-squares measure is sensitive to sample dimensions (e.g., the number of industries, the number of businesses per industry), which limits its usefulness as effect size measure. For example, the industry sum-of-squares measure will be higher with stronger industry effects or with more industry degrees of freedom. It follows that we obtain a different sum of squares when changing the degrees of freedom (e.g., by picking a sample with different dimensions) even when the strength of industry effects is held constant. Using 25 samples from nine empirical studies (N = 212,112), I find that this sensitivity is strong in practice: knowing only the dimensions of a sample is sufficient to predict well the sum-of-squares measure.

To assess the explanatory power of industry, corporate, and business effects, I conduct a meta-analysis using not the sum-of-squares measure of effect size, but instead the variance and standard deviation measures. Using 18 samples from 16 studies (N = 225,183), the variance findings are 0.08 for industry, 0.14 for corporate, and 0.36 for business effects; the standard deviation findings are 0.28 for industry, 0.36 for corporate, and 0.59 for business. Thus, business effects explain about twice what industry and corporate effects do, and corporate effects explain somewhat more than do industry effects.

These results contribute in two ways to answering the question of how much of performance differences is explained by each of these factors. First, the effect size for business is substantially greater than for industry and corporate, but that difference depends on the measure employed. For example, the average effect size for business is more than four times that of industry when measured in variances, but only twice as large when measured in standard deviations. Indeed, under the standard deviation measure, industry and corporate effects together explain as least as much as do business effects. The variance measure is based on squared distances, so differences between factors are amplified (Brush and Bromiley 1997, Hunter and Schmidt 2004); taking the square root of the standard deviation measure reduces that amplification. Thus, the question of explanatory power is as much a question about the strength of particular effects as it is about the effect size measure itself.

Second, corporate effects are somewhat greater than industry effects. This finding does not depend on whether the variance or instead the standard deviation measure is used. Two reasons may explain why this result has not previously been established in the literature. First of all, the corporate effect is greater than the industry effect in most samples, but not in all samples. It is more difficult to separate signal from noise in a single sample than in a large set of samples, as is employed here. Secondly, the industry factor has received more attention than the corporate factor. All studies in the empirical literature include industry effects, but several do not include corporate effects. In the theoretical literature, most debate concerns the merits of adopting an external (industry) perspective versus an internal (business) perspective rather than a corporate perspective. Hence this finding reinforces the relevance of corporate effects for the understanding of performance differences.

2. Measures

2.1. Background

The empirical approach to explaining firm performance amounts to describing at which level performance differences occur. Here “performance” is usually measured as return on assets and less frequently as return on sales, return on invested capital, or market share. The levels investigated most often are industry, corporate parent, and business. An industry is the set of businesses that sell similar goods or services. Similarity is often captured via an industry classification, such as the Standard Industrial Classification (SIC) or its successor, the North American Industry Classification System (NAICS). Thus we can distinguish among the industries of rail transportation (NAICS 4821), insurance carriers (NAICS 5241), and electric power generation, transmission, and distribution (NAICS 2211). A corporate parent is a legal entity that owns and operates one or more businesses (Rumelt 1991).¹ For example, Berkshire Hathaway owns and operates many businesses, including a rail transportation business, an insurance business, and an electric power generation business. The businesses of a corporate parent are demarcated such that each business operates in one and only one industry. It follows that a business is interpreted from a market perspective (i.e., as a business segment) and not from an organizational perspective (i.e., not as a business unit) (McGahan and Porter 1997). Even if the operations of rail transportation and electric power generation were organized in a single business unit, they would still count as two businesses for the purposes of performance analysis. Insurance operations that are distributed between two business units would still be considered a single business.

Figure 1 gives examples of performance differences occurring at different levels. Consider four businesses, each belonging to one of two corporate parents (ovals in the figure) and one of two industries (triangles). Our aim is to characterize the occurrences of high-performing businesses (gray squares) and low-performing businesses (white squares). Therefore, explaining “firm” performance is equivalent to explaining the performance of businesses, which belong to corporate parents and industries. Industry effects are such that performance differences occur between industries. In Example I, all high-performing businesses are in the industry on the figure’s left side while all low-performing businesses are on the right-side industry. Corporate effects are such that performance differences occur between corporate parents. So in Example II, all high-performing businesses belong to the left-side corporate parent and the low-performing businesses to the right-side corporate parent. Business effects are such that performance differences are related to the specifics of the business. In Example III, the leftmost and rightmost businesses are high-performing, while those in the middle are low-performing. This pattern cannot be explained either by industry or corporate effects because each corporate parent and each industry has both a low-performing and a high-performing business.

These examples are simplifications. First of all, they are “pure”examples: in each case, a single factor accounts fully for the pattern of performance differences. In reality, the three effects co-occur; the result is complex performance patterns—especially when considering more than two industries, two corporate parents, and four businesses. Second, these examples ignore time. If we interpret Figure 1 as performance from a single year, then all the patterns may reflect nothing more than randomness. Only if these performance differences persist over longer periods can we meaningfully talk about industry, corporate, and business effects.

2.2. Basic Properties of Measures

Three different measures have been used to quantify the effect of year, industry, corporate, and business effects on performance. The first measure is based on the sum of squares of these effects (e.g., Goddard et al. 2009, Ma et al. 2013, Fitza 2014, Zacharias et al. 2015); the second on their variances (e.g., Chan et al. 2010, Karniouchina et al. 2013), and the third on their standard deviations (e.g., Brush and Bromiley 1997, Hough 2006). The sum-of-squares and variance measures have been used most frequently and the standard deviation measure least frequently.

These measures are related but usually yield different outcomes, which precludes aggregation. In general, a sum of squares is the sum of squared differences from a mean. A sum of squares divided by the degrees of freedom gives the variance, and the square root of a variance is the standard deviation. The sum of squares measure does not account for degrees of freedom. A larger sample yields a higher sum of squares but not a higher variance or higher standard deviation. Formally, we have the following:

To describe industry, corporate, and business effects, we can write the performance (p_jkt) of a business in corporation k, industry j, and year t in line with Rumelt (1991) as

Here m is a constant (it signifies the mean performance of all businesses across all years); Y_t is the year effect for year t (i.e., the performance difference in year t relative to the mean); I_j is the industry effect for industry j (i.e., the performance difference in industry j relative to the mean); C_k is the corporate effect for corporation k (i.e., the performance difference of corporation k relative to the mean); B_jk is the business effect for a business in corporation k and industry j (i.e., the performance difference between that business relative to the mean); and e_jkt is an error term. For simplicity, the model does not include interaction effects (in addition to the business effect, which can be viewed as an interaction between the industry and corporate effects).

For industry, corporate, and business effects, calculation of the three measures is the same when transforming variance to standard deviation (the square root) but differs when transforming the sum of squares to variance (i.e., dividing the industry sum of squares by its degrees of freedom does not yield the industry variance). However, the general intuition holds. The sum of squares measure is sensitive to sample dimensions, e.g., the number of businesses per industry, and the variance and standard deviation are not (or much less so).

The total sum of squares (SST) is the sum of squared differences from the performance mean. The goal with this approach is to allocate the total sum of squares among the sum of squares of the effects: year, industry, corporate, and business. For example, the sum of squares industry (SSI) is that part of the total sum of squares arising from performance differences between industries, and the sum of squares business (SSB) is the part arising from performance differences between businesses. For comparison, effect sizes are expressed as a proportion of the total sum of squares: SSI/SST and SSB/SST. These ratios correspond to a regression’s R² values that are due to (respectively) industry and business effects.

The sum-of-squares effect size measure depends not only on the strength of the effects but also on the sample dimensions (e.g., the number of industries, the number of businesses per industry). Even if the sum of squares business is greater than the sum of squares industry, we cannot then say that business is more important than industry because the difference could be due to the sample’s dimensions and not to any difference in the relative strength of their effects. This property limits the measure’s usefulness for aggregating results across samples, which is the goal here.

I illustrate the influence of sample dimensions with a simulation.² For simplicity (but without loss of generality), I focus on industry and business effects and ignore corporate and year effects. Let the performance (p_jlt) of business l in industry j and year t be written as

By construction, industry effects I_j and business effects B_jl are equally important in determining performance: both are normally distributed with mean 0 and standard deviation 1. The error (e_jlt) has the same distribution. The overall mean (m) is 0. The range of sample dimensions matches that of the studies included in the meta-analysis: the number of industries ranges from 5 to 2,375 and the number of businesses per industry from 2 to 167. The number of years is fixed at eight, which is the median of all the studies considered.

Most researchers use ANOVA for the sum-of-squares measure; I shall do likewise. For a given combination of parameters, each industry has the same number of businesses. The result is a balanced design (i.e., an equal number of observations per business and an equal number of businesses per industry). The sum of squares can therefore be calculated directly from the data without the need for regression analysis (which would give the exact same results but takes much longer to calculate) (Searle 1971).³ For the sum-of-squares formulas, see column SS of Table 1 (or Kutner et al. 2005, pp. 1096–1099).

Table 1: Sum of Squares and Variances for Simulation

Figure 2 plots the explained sum of squares for industry and for business (SSI/SST and SSB/SST). Each data point represents the average of 1,000 randomly generated samples. In the figure’s right panel, the number of business per industry is fixed at the studies’ median (10); in the left panel, the number of industries is fixed at the studies’ median (69). These results illustrate two points. First, the explained sum of squares for industry and business effects depends on the sample dimensions. Second, despite the equal importance (by design) of industry and business effects, their sums of squares differ for most parameter combinations.

The right panel of Figure 2 shows that, with two businesses per industry, business effects explain only about 19% of performance while industry effects explain 51%. Yet with 167 businesses per industry, the situation is reversed: business effects explain more than do industry effects (38% versus 33%). The only difference in these two cases is the number of business effects, not their size. In the left panel, if there are five industries, then industry effects explain about 29% of performance, while business effects explain 38%. Yet for a higher number of industries, industry effects explain more than business effects: when there are 100 industries, the respective percentages are 37% and 34%. The only difference is that the sample size has increased. So if the sum of squares is taken as an indicator of importance for performance, then one could mistakenly conclude that industry matters less than business if few industries are sampled, and more if many are sampled.

This problem is not resolved by using adjusted R², which penalizes the addition of more variables, instead of R² (cf., Brush et al. 1999). I calculate the adjusted R² by adjusting the sum of squares with the appropriate degrees of freedom (see Figure 3). The derived values for adjusted R² are somewhat lower than those derived for R² (because of the penalty just described), but they lead to the same conclusion: notwithstanding equal industry and business effects, their adjusted sum of squares differ markedly for most parameter settings and are sensitive to sample dimensions.

The problem arises as follows. Changing the degrees of freedom (e.g., the number of industries) will naturally lead to a change in the sum of squares (e.g., SSI), simply because a different number of squares need to be summed (see Table 1). At the same time, the total sum of squares (SST) will change, but not by the same factor. Hence, the ratio of the two will change (SSI/SST).⁴ As I will show next, this problem is much diminished (or entirely absent) for ratios of variances or standard deviations.

The total variance is the variance in performance. The goal here is to allocate it among the variances of the effects: year, industry, corporate, and business. For purposes of comparison, the effect size is given as a proportion of the total variance—for example, ${\sigma }_I^2/{\sigma }_p^2$ for industry or ${\sigma }_B^2/{\sigma }_p^2$ for business. Unlike the sum-of-squares measure, the variance measure is not (or is only weakly) dependent on sample dimensions.

To illustrate this difference, I calculate the variance measure for the previously simulated data (which, by design, had equally strong industry and business effects). Using the simplified performance Equation (3), which includes industry and business effects but not year and corporate effects, we can write (see e.g., Rumelt 1991)

Assuming independent effects, this expression decomposes the total variance (${\sigma }_p^2$) into the variances of the effects (${\sigma }_I^2$ for industry and ${\sigma }_B^2$ for business), and the variance of the error (${\sigma }_e^2$). Following the extant literature, I use a random-effects assumption when calculating the variances of the effects. Hence, these are expressed as population variances (${\sigma }_I^2$ and ${\sigma }_B^2$) and not as sample variances ($s_I^2$ and $s_B^2$), which would apply with a fixed-effects assumption. However, the discussion that follows holds for both random- and fixed-effects.

For the variance measure, most researchers use either variance component analysis (VCA) or hierarchical linear modeling (HLM). The key difference between these methods is in how they account for multilevel data and the relationships between levels (Hough 2006, Misangyi et al. 2006). This distinction is less relevant to the simplified model here, which includes only industry and business effects. The data are nested (i.e., a business belongs to an industry) but not cross-classified (i.e., a business does not belong to an industry and a corporation). Because of ease of calculation, I use VCA to estimate the variances (Searle 1971). This procedure involves equating the calculated sum of squares with their expected values and then solving the system of equations.⁵

Figure 4 plots the average explained variance for industry and business (${\widehat{\sigma }}_I^2/{\widehat{\sigma }}_p^2$ and ${\widehat{\sigma }}_B^2/{\widehat{\sigma }}_p^2$); each explains about a third of the variance in performance. The number of businesses has no effect (right panel), but the number of industries has some effect when there are few industries (left panel). This is because it is difficult to estimate precisely the variance of a factor when limited levels are sampled (Kutner et al. 2005, Hox 2010).⁶ That difficulty has been referred to as the problem of “data sparseness” (for further discussion, see Stavropoulos et al. 2015). Note that this problem arises only for a small number of levels and appears to be modest in size compared to the sum of squares. Therefore, provided the standard errors of the effects’ estimates are not too high, the variance measure will be insensitive to sample dimensions.

To compare the sum-of-squares and variance measures, Figure 5 shows the effect sizes for industry relative to business (SSI/SSB and ${\widehat{\sigma }}_I^2/{\widehat{\sigma }}_B^2$). The variance measure attributes roughly equal strength to industry and business, irrespective of the sample dimensions; in contrast, the sum-of-squares measure’s ranking depends on the sample dimensions.

The standard deviation measure is the square root of the variance measure. Expressed as a proportion of the performance standard deviation, we have, for example, σ_I/σ_p for industry and σ_B/σ_p for business. Like the variance measure, the standard deviation measure is relatively insensitive to sample dimensions.

The variance measure is based on squared distances, so differences in linear distances are amplified (Brush and Bromiley 1997, Hunter and Schmidt 2004). The square root of the standard deviation measure reduces that amplification.⁷ To illustrate this dynamic, I intentionally make the business effects stronger than the industry effects. Thus, I simulate data in which the business effects have been multiplied by a factor that ranges from 1 to 3 while leaving industry effects unchanged. The number of industries is fixed at the median (69), as is the number of businesses per industry (10); the other settings remain unchanged. I use VCA to obtain the variances and then take the square root to obtain the standard deviation.

Figure 6 illustrates the effect size for business relative to industry using the variance and the standard deviation measures (${\widehat{\sigma }}_B^2/{\widehat{\sigma }}_I^2$ and ${\widehat{\sigma }}_B/{\widehat{\sigma }}_I$, respectively). Making the business effect three times as strong yields an effect size—for business relative to industry—of three (with the standard deviation measure) and more than 10 (with the variance measure). Thus, the variance measure amplifies differences between industry and business, making industry seem less important than it actually is. These results reflect the variance measure’s use of squared distances. Because the sum-of-squares measure is also based on squares, it likewise displays such amplification.

2.3. Methods

The literature has active discussions on fixed- versus random-effects assumptions and on methods. Here I describe how these discussions relate to the three measures.

The year, industry, corporate, and business effects can be described as random or fixed (Searle 1971). If random, then the effects are seen as coming from a larger population. If fixed, then the effects are not seen as coming from a larger population. For instance, with a random-effects assumption, we can describe industry effects in terms of the population standard deviation (σ_I)—that is, how much all industries differ from each other. With a fixed-effects assumption, we can describe such effects only in terms of the sample standard deviation (s_I), or how much industries in the sample differ from each other.⁸ Because it is not always obvious whether effects are fixed or random (Searle 1971), some variance decomposition studies have used fixed effects (e.g., Mackey 2008, Goddard et al. 2009, Ma et al. 2013) and others have used random effects (e.g., Roquebert et al. 1996, Chang and Singh 2000, Chan et al. 2010). These types have been used in separate models (e.g., Schmalensee 1985, Rumelt 1991, McGahan and Porter 1997) and also in the same model, as when some effects are fixed and others are random (i.e., mixed-effects) (e.g., Hough 2006, Misangyi et al. 2006).

The three measures can be calculated for both fixed and random effects (see Table 2). The sum-of-squares measure is the same for fixed and random effects. Sum of squares is a property of a sample; hence, the presence or absence of a larger population is irrelevant. The variance and standard deviation measures differ for random versus fixed effects: for random effects, these measures refer to the population variance and standard deviation; for fixed effects, they refer to the sample variance and standard deviation.⁹

Table 2: Effect Size Measures for Fixed and Random Effects

Even though the fixed versus random effects assumption does not limit one’s choice of measure, in practice, however, nearly all studies have used either a fixed-effects assumption with the sum-of-squares measure or a random-effects (or mixed-effects) assumption with the variance and standard deviation measures (an exception is Brush et al. 1999). Table 3 illustrates this mapping between the effects assumption made and the measures used. It shows also the variety of methods employed, including analysis of variance (ANOVA), hierarchical linear modeling (HLM), two-stage least squares (2SLS), and variance components analysis (VCA). In the literature, debate has centered on the appropriateness of the fixed- versus random-effects assumption (columns in the table) and on the merits of the different methods (cells in the table). For a discussion, see for example Brush and Bromiley (1997), McGahan and Porter (2002, 2005), Ruefli and Wiggins (2003, 2005), Hough (2006), and Bou and Satorra (2010). Here the discussion has focused on the measures (rows in the tables).

Table 3: Methods for Fixed, Mixed, and Random Effects

3. Methods

3.1. Identification of Studies

Several steps were taken to identify studies for the meta-analysis. First, the Business Source Premier database and the Google Scholar database were searched using the following search terms: industry effect, corporate effect, or business effect; or variance, decomposition, and at least one of industry, corporate, and business. Second, the following journals were searched for any study containing at least two of industry, corporate, or business (anywhere in the text): Academy of Management Journal, Administrative Science Quarterly, Global Strategy Journal, Journal of Business Research, Journal of International Business Studies, Journal of Management, Journal of Management Studies, Long Range Planning, Management Science, Organization Science, Organization Studies, Strategic Management Journal, Strategic Organization, and Strategy Science. Third, all studies citing the foundational study of McGahan and Porter (1997) were searched for the word “variance.” Fourth, a request for unpublished (and published) studies was sent to strategy scholars via the Business Policy and Strategy listserv (BPS Net). Fifth, all works cited in the identified studies were screened for their possible relevance.

Most meta-analyses in the strategy literature use as inputs sample estimates that do not depend on modeling choices (e.g., pairwise rather than partial correlations). Such model-free estimates are unavailable here. To ensure comparability among studies, I selected those that employ similar models (Hunter and Schmidt 2004). A study was included in the meta-analysis only if it met the following conditions. First, it had to include industry, corporate, and business effects. Second, it had to report an effect size in standard deviations, variances, or sums of squares (two studies excluded). Third, the study had to describe a model with no more than one interaction term involving year, industry, or corporate effects (in addition to the business effect) (three studies excluded). Fourth, for VCA or HLM, the study had to report a model without covariances (two studies excluded).

3.2. Selection of Analyses

Most studies report multiple analyses. For each study, only one is included per measure by sequentially applying the following criteria. First, for overlapping samples, choose the largest. Hence, aggregate samples are chosen instead of subsamples of select industries, but also samples are selected such that single-business corporations are included rather than excluded. The latter selection may affect the estimates, especially those of the corporate effects, per Bowman and Helfat (2001). I investigate the impact of including versus excluding single-business corporations in an additional analysis. Second, for the model, choose one without additional interaction effect (if one is provided). Third, for the method, choose HLM over VCA (only one study uses both). Note that the more recent studies reporting variances or standard deviations use HLM.

3.3. Calculation of Effect Sizes

Few studies report standard deviations; moreover, in those that do report them, the standard deviations of effects are relative to each other rather than to performance. So even for studies that do report standard deviations, I rely on the reported variances to calculate the standard deviations used here (by taking the square root). Sum-of-squares measures could conceivably be converted into variances using Henderson’s (1953) “method III,” but studies do not report data sufficient for conducting these calculations. For this reason, the sum-of-squares samples are reported separately.

The procedures just described yielded, for the standard deviation and variance measures, a set of 16 studies reporting on 18 samples with a total of N = 225,183 business-year observations. For the sum-of-squares measure, I obtain a (partially overlapping) set of nine studies reporting on 25 samples with a total of N = 212,112 observations.

3.4. Nonindependence

For a given sample, the effect sizes are not independent. For example, the business standard deviation (as a ratio of the total standard deviation) depends on the corporate standard deviation. To account for such nonindependence, I treat the data as paired when testing for differences in effect size. For example, I analyze the distribution of the business minus the corporate standard deviation across samples.

Across samples, the effect sizes are unfortunately also not independent. This violates a key assumption of a meta-analysis. The problem arises because many samples are from the United States (12 of 18 variance and standard deviation samples). Most of these U.S. samples are from the Compustat database, and some with overlapping time periods. To address this non-independence, I provide an analysis restricted to the six non-U.S. samples, which draw from different databases and cover different regions and time periods. Because the results are broadly consistent, I report all samples as the main analysis and the non-U.S. samples as an additional analysis. Note that the reported confidence intervals for the main analysis are merely indicative and are probably too narrow.

4. Results

Table 4 lists the studies and samples. The three effect size measures are also reported; these measures are standardized relative to performance (e.g., industry/performance). The samples are sorted by industry effect size for the variance or standard deviation or (absent those measures) for the sum of squares. Three summary statistics are reported in the bottom rows. The sample size–weighted mean puts more weight on larger samples, whereas the unweighted mean gives equal weight to all samples; the median reduces the influence of any outliers.

Table 4: Studies, Samples, and Their Effect Sizes

4.1. Sum of Squares vs. Variance

The sum-of-squares measure for actual samples is sensitive to sample dimensions. In Figure 7 the sum-of-squares samples (1, 14, 19–41) are represented by dots.¹⁰ On the vertical axis is the explained sum of squares per factor; on the horizontal axis are the degrees of freedom used per factor as a proportion of the total degrees of freedom. If the degrees of freedom are not reported, then these are calculated for year, industry, and corporate effects as (respectively) the number of years minus 1, the number of industries minus 1, and the number of multibusiness corporations minus 1. For business effects, missing degrees of freedom are approximated as the number of businesses minus the number of industries minus the number of multi-business corporations plus 1. In line with the simulations, more relative degrees of freedom correspond to a higher explained sum of squares for a factor. For instance, a sample with more relative industry degrees of freedom displays (on average) a greater industry effect whereas a sample with more relative corporate degrees of freedom displays a greater corporate effect.

The variance measure is much less sensitive to sample dimensions. The correlation between relative degrees of freedom and effect sizes is weak for the variance but is strong for the sum-of-squares measure (see the df columns in Table 5).¹¹ The “data sparseness” problem mentioned before with respect to the variance measure may manifest itself not with relative but rather with absolute degrees of freedom (Stavropoulos et al. 2015). However, correlations between effect sizes and the number of years, industries, multibusiness corporations, and businesses (column n_k) or their natural logarithms (column ln(n_k)) remain weak.

Table 5: Sample Dimensions Strongly Correlate with Sum of Squares But Not with Variance

In the two samples that provide both measures, the effect sizes of the sum-of-squares measure differ noticeably from those of the variance measure (see Figure 8). In sample 1, the corporate effect’s size as measured by sum of squares is more than twice its size as measured by variance. In fact, using sums of squares shows corporate effects to be greater than industry effects whereas the opposite result obtains when variances are used. In sample 14, the ranking of effects is preserved; however, the industry effect size under sum of squares is more than double that under variance while the corporate effect size is less when the sum-of-squares measure is used.

At the aggregate level, differences are more subtle. The weighted mean for the sum of squares measure is similar to that for the variance measure (see Table 6). This outcome is mainly driven by samples 31 and 35, which together account for more than 60% of all sum-of-squares observations. Looking instead at the unweighted mean, we see a notable difference for industry effects (0.127 versus 0.090). These findings underscore the importance of using a common effect size measure.

Table 6: Meta-Analytic Results for Variance and Standard Deviation for k = 18 Samples

4.2. Variance vs. Standard Deviation

For the variance and standard deviation samples (1–18) results are consistent across the three summary statistics (weighted mean, unweighted mean, median). I will focus on the weighted mean as an estimate of a population parameter (Hunter and Schmidt 2004).

The weighted mean for the variances are 0.01 for year, 0.08 for industry, 0.14 for corporate, and 0.36 for business effects. The weighted mean for the standard deviations are 0.08 for year, 0.28 for industry, 0.36 for corporate, and 0.59 for business effects.¹²Figure 9 and Table 6 report meta-analytic results with bootstrapped 95% confidence intervals based on 10,000 replications. We can informally interpret these variances and standard deviations as follows. An effect is defined as a performance deviation from an overall mean, and the estimates give an indication of the relative size of these performance deviations associated with each factor. On average, then, the deviations associated with corporate are somewhat greater than those associated with industry, and those associated with business are substantially greater. A formal interpretation would reference Equation (2), where performance is the sum of four factors (and a mean and an error term). The effect sizes provide estimates of the distributions of these factors. For example, the industry effects can be seen as coming from a distribution with mean 0 and a variance of 0.08 or a standard deviation of 0.28. The year, corporate, and business effects each have their own mean 0 distribution with variance or standard deviation as previously reported.

Figure 10 shows the differences for variances (left panel) and standard deviations (right panel) between business and corporate effects, between corporate and industry effects, and between industry and year effects (for numbers, see Table 6). The figure plots the weighted mean differences and the 95% confidence intervals. No confidence interval overlaps with 0. Hence, business effects are the strongest, followed by corporate, then industry, and finally year effects.

Thus, the results are qualitatively similar for variance and standard deviation. In line with the simulation, the variance amplifies differences between factors. For the variance, the industry effect is ×10.5 (standard deviation: ×3.5) the year effect, the corporate effect is ×1.6 (standard deviation: ×1.3) the industry effect, and the business effect is ×2.6 (standard deviation: ×1.6) the corporate effect.

4.3. Additional Analyses for Standard Deviation

To determine how much of the differences between samples are due to sampling error or differences in underlying effects, we would need the standard errors of the sample estimates. These standard errors are neither reported nor can they be derived from the data provided. Instead, I explore the extent to which estimates differ by the following characteristics: sample, method, and model (see Table 7 and Figure 11). I report here the results for standard deviation (the results for variance are qualitatively similar).

Table 7: Meta-Analytic Results for Standard Deviation by Subgroup

In Panel A of Figure 11 (and in rows A1 and A2 of Table 7), the samples are split by region, which refers to the corporate parent’s location. Most samples include the international businesses; for example, U.S. samples contain businesses that operate beyond U.S. borders. The results are fairly similar, although industry, corporate, and business effects are all somewhat lower in non-U.S. than in U.S. samples. The lack of substantial differences reduces concerns about the possible nonindependence of the U.S. samples.

Due to data limitations, Rumelt (1991) restricted his analysis to manufacturing firms only. Nowadays, most data sets include nonmanufacturing firms, too. Out of 18 samples, four are manufacturing firms only. The results of these four samples are similar to those of the other 14 (see Panel B of Figure 11 and rows B1 and B2 of Table 7).

For a single-business corporation, the business and corporate effects are indistinguishable. Some studies exclude such corporations whereas others include them. If they are included then, under VCA, an explicit assumption is needed. The convention in this regard is to estimate a business effect and then set the corporate effect to zero. This approach leads to underestimating the corporate effect and overestimating the business effect (Bowman and Helfat 2001). Under HLM, no such explicit assumption is required because the model can be estimated. It is difficult ex ante to state with high confidence whether or not the inclusion of single-business corporations leads to biases in the corporate or business effect. Our current knowledge on biases is based on simulations, not on analytical results (e.g., Baldwin et al. 2011). These simulations indicate that biases are small or nonexistent when the total number of groups (i.e., single-business plus multibusiness corporations) is high, even if the percentage of “singletons” (single-business corporations) is high. The term “high” is 168 groups consisting of 57% singletons in Clarke and Wheaton (2007) and 500 groups consisting of 70% singletons in Bell et al. (2008, 2010). One reason for cautious optimism then is that the HLM samples have many corporations: even the smallest sample contains 136 corporations, and the second smallest has 998. Because of the different approaches for VCA and HLM, the comparison here is within method (i.e., either VCA or HLM). In Section 4.3.4, the comparison is between methods (i.e., VCA versus HLM).

Only one VCA sample includes single-business corporations. Its corporate effect (sample 18: 0.084) is, as anticipated, the lowest across all samples; it is also substantially below the sample with the second-lowest VCA sample (sample 8: 0.225). In contrast, only one HLM sample excludes singletons. Its corporate effect (sample 17: 0.550) is the highest across all samples and substantially above the second-highest HLM sample (sample 14: 0.449). The sample without single-business corporations differs from the others not only in sample selection but also in model specification: it views the corporate effect as consisting of a business-invariant and a business-variant component. Hence, from this single and atypical sample, we cannot infer the impact of single business corporations, when using HLM.

Among the 18 samples, 11 use VCA and seven use HLM. Panel C of Figure 11 (and rows C1 and C2 of Table 7) show that industry effects are somewhat lower and that corporate effects are somewhat higher with the HLM than with the VCA method. As a result, the difference between industry and corporate effects is more pronounced under HLM. One distinction is that the VCA samples typically exclude single-business corporations, whereas these are included in the HLM samples. Yet, in light of the simulation results mentioned previously, this distinction may not actually explain the differences in effects. Furthermore, if HLM with single business corporations overestimates the corporate effect, then we should expect it to underestimate the business effect; but the business effect is, if anything, greater under HLM than under VCA. Thus, further investigation comparing these two approaches is needed.

Model specifications differ across samples. In particular, half of the samples employ models with only year, industry, corporate, and business effects (“YICB only”). Studies in the other half also include such terms as country, region, and/or an interaction effect between industry and year. Panel D of Figure 11 (and rows D1 and D2 of Table 7) show that the year, industry, and corporate effects differ little across alternative model specifications. The business effect becomes weaker when additional terms are included, which might be explained by the business effect picking up influences that are fixed for a business, but vary across industry or corporations. For example, a business may operate in a single region even as its industry and corporation span multiple regions. In that case, omitting region from the specification will lead to a higher business effect.

Thus, a consistent pattern emerges across alternative samples, methods, and models: the industry effect is about half that of the business effect, and the corporate effect is slightly greater than the industry effect.

5 Discussion

Based on Cohen’s f² (1988) criteria yielding 0.02, 0.13, and 0.26 for (respectively) small, medium, and large explained variance, we can classify the effect sizes for industry and corporate as “medium” and for business as “large.”¹³ There are two striking aspects of the findings reported here. First, business effects explain the most, but their explanatory power, relative to industry and corporate effects, depends on whether the variance measure or instead the standard deviation measure is used. Second, the effect size for corporate effects is somewhat greater than for industry effects; that relation has not been well established in existing research, regardless of the measure used.

When analyzing industry, corporate, and business effects, one should bear three cautionary statements in mind. First, the size of an effect does not equal its importance. A small performance difference can be enough to spell the death (or survival) of a business, and a small difference in return on assets may represent a big difference in absolute returns. Second, size of an effect is not the same as its influence (Bowman and Helfat 2001, McGahan and Porter 2005). Thus, an effect does not, in itself, reveal the managerial actions required to generate the performance difference. For example, if a successful corporate parent consistently picks profitable industries to enter, then this upside will be viewed as an industry effect rather than as a corporate effect. In other words, the empirical approach identifies correlates, not causes, of performance.

Third, it follows that the observed effects are not causal effects. The literature on variance decomposition defines an “effect” as a performance deviation from a mean (Rumelt 1991, McGahan and Porter 1997)—for example, the mean performance of the businesses of one corporate parent relative to an overall mean performance. Both types of performances are observable. In contrast, a causal effect is interpreted as the difference between factual and counterfactual performance (Rubin 1974, Morgan and Winship 2015); an example here is the performance of a business of a given corporate parent relative to the performance of the same business if it were under different ownership. By definition, factual and counterfactual performances cannot be observed simultaneously. So then, what can be learned from this empirical approach? Most importantly, it offers a set of stylized facts (McGahan and Porter 2005). If the field of strategy seeks to explain differences in firm performance, then we need to identify those differences and the level at which they occur.

This study has the following implications for the choice of effect size measure. The variance and standard deviation should be favored over the sum of squares as an effects measure. “An effect-size measure is a standardized index and estimates a parameter that is independent of sample size” (Olejnik and Algina 2003, p. 434)—and, I would add, independent of sample dimensions. The sum-of-squares measure does not satisfy this criterion. Chang and Singh (2000) show that the level of industry aggregation (e.g., three- versus four-digit industry classification) matters. However, the argument here is that, for a given level of industry aggregation, the sum of squares is sensitive to sample dimensions. As mentioned previously, sum-of-squares measures are well predicted using only the number of years, industries, corporations, and businesses in the sample. The unfortunate consequence is that relative effect sizes can differ between samples simply because of their different dimensions. For this reason, the preference of variance and standard deviation over sum of squares is clear.

The variance measure is (mostly) insensitive to sample dimensions, so the choice between the standard deviation and variance measures is more subjective. One downside of the variance measure is that large effects are amplified and small effects are compressed, which may reduce the latter’s perceived importance. For example, the weighted average variance for year effects is less than 0.01. Most scholars would be reluctant to claim that the year is irrelevant, but this is what the variance measure seems to suggest. Similarly, the weighted average variance for industry is only 0.08, which could create the false impression that industry does not matter. At a minimum, it makes industry appear to matter less than it actually does. For instance, one of the most popular strategy textbooks notes that “[i]t appears that industry environment is a relatively minor determinant of a firm’s profitability. Studies of the sources of interfirm differences in profitability have produced very different results […] but all acknowledge that industry factors account for a minor part (less than 20%) of variation in return on assets among firms” (Grant 2016, p. 90). Although the author then proceeds to defend industry analysis, it is unclear whether a defense is needed (recall that, under Cohen’s criteria, a small effect is around 2% and a medium effect around 13%). One upside of the variance measure is that it has a long tradition in the social sciences, which facilitates comparability.

This study suggests two opportunities for further research in this area. First, it was found that the ranking of the industry, corporate, and business effects was fairly constant across samples, methods, and models (using the variance or standard deviation measure). Even so, each factor separately exhibited variability across studies. A subgroup analysis was used here to explore that variability. Future studies can analyze the same question using individual samples and possibly subsamples. The second research opportunity is that—given the robust findings on industry, corporate, and business effects—it would be interesting to identify which industries, corporations, and business are overperformers and which are underperformers. We could then move from a factor to an individual effect (e.g., from the corporate factor to a specific corporation).