Mapping Market Structure Evolution

1. Introduction

Firms need to understand the competitive structure of their market to develop effective strategies and create sustainable competitive advantages (Hunt 1983, Rao and Sabavala 1986). Such knowledge allows managers to assess how well their offerings are positioned, develop clear positioning strategies, and identify critical elements of their strategies (Lilien and Rangaswamy 2004). Beyond positioning, market structure analysis informs new product development, product policy, competitive advertising, and pricing strategies (Urban et al. 1984). Moreover, broader strategic considerations, such as defining a firm’s business, assessing threats and opportunities, and allocating resources, also require a profound understanding of competition in a market (Day et al. 1979). Hence, market structure analysis is an essential ingredient in the strategic market-planning process (Day 1984).

A common element of market structure analysis is the derivation of a market structure map, that is, a spatial representation of firms’ competitive positions relative to one another based on some measure of their competitive relationships (DeSarbo et al. 1993). Market structure maps visually summarize large amounts of information, which facilitates decision making; helps managers think more strategically; and provides them with explorative tools to identify competitors, discover submarkets, and guide positioning and differentiation decisions (Smelcer and Carmel 1997, Lilien and Rangaswamy 2004).

With the increasing availability of extensive data on firm positioning and consumer perception, market structure maps’ informative potential continues to grow. Recent studies in marketing leverage various data sources, such as consumer search, shopping baskets, product reviews, and social media engagement, to create market structure maps (Kim et al. 2011, Lee and Bradlow 2011, Netzer et al. 2012, Tirunillai and Tellis 2014, France and Ghose 2016, Ringel and Skiera 2016, Gabel et al. 2019, Liu et al. 2020, Yang et al. 2022). These maps are typically static; that is, they typically represent an isolated snapshot of a market’s structure in a single period. Yet, changes in consumers’ needs and preferences, new competitors' entries, or incumbents' repositioning lead to evolving rather than static market structures (Lambkin and Day 1989, D'Aveni 1994, Ning and Villas-Boas 2021).

Recognizing that market structures tend to evolve rather than remain static (Elrod et al. 2002), earlier marketing research proposed approaches to examine market structures across more than one period (DeSarbo and Carroll 1985, Moore and Winer 1987, Cooper 1988, Mela et al. 1998). However, the application of these earlier approaches in the more recent marketing literature is rare—even though many studies emphasize the potential value of monitoring the evolution of market structure over time (e.g., Kim et al. 2011, Lee and Bradlow 2011, Netzer et al. 2012, Ringel and Skiera 2016, Wedel and Kannan 2016, Gabel et al. 2019). A possible reason might be that, as we later show, mapping the evolution of market structure in today’s large and fragmented markets is not easily possible with existing approaches.

We suggest that mapping the evolution of market structure across multiple periods provides additional insights beyond those of single snapshots in time. These additional insights, elaborated in what follows, emerge from what we call firms’ trajectories—the evolutionary paths of firms’ positions over time relative to all other firms in a market (hereafter referred to as “firms’ trajectories” for brevity). In contrast to merely describing the change in firms’ positions from one snapshot of market structure to another, trajectories capture underlying trends in competitors’ positions over multiple periods. The joint examination of firms’ trajectories in a market reveals the evolution of its structure and offers a unique forward-looking perspective on how market structure might look in the future.

Specifically, we suggest that firms’ trajectories can contribute the following insights to market structure analysis:

1. A forward-looking perspective on competition: By comparing other firms’ trajectories with their own firm’s trajectory, managers can better anticipate their competitive situation in the future. Whereas converging trajectories indicate competitive threats, diverging trajectories point to an increase in differentiation. Likewise, new entrants’ trajectories reveal where they seek to position themselves in the market, thereby enabling incumbents to (a) better qualify the competitive threat that these new entrants might pose and (b) do so at an early stage. Finally, a more gradual convergence of multiple firms’ trajectories away from their previous competitors can point to a new submarket’s emergence.

2. A lens on competitors’ strategic intentions: Sudden changes in other firms’ trajectories can point to repositioning efforts of those firms. By monitoring other firms’ trajectories, managers can detect (or see the impact of) such repositioning, which they may need to account for in their own strategy formulation.

3. An indicator of strategy effectiveness: Any change in a firm’s trajectory should ideally align with the firm’s strategic objectives. Therefore, monitoring their firm’s trajectory enables managers to better evaluate the effectiveness of their marketing strategy. Moreover, doing so helps to assess the impact of marketing actions, such as (re)positioning efforts or marketing mix decisions (i.e., whether they “moved the needle”).

Consider, for example, a scenario where a change in strategy prompts a firm’s management to reposition the firm (e.g., change the firm’s name, introduce a new logo, or adjust the marketing mix). Such changes are common, as recent examples from the tech sector illustrate (e.g., Odeo/Twitter, Netflix, or Facebook/Meta). Because repositioning usually requires substantial investments, managers must monitor the impact and effectiveness of their efforts. Likewise, investors likely want to evaluate whether the current management achieves the desired transition to a new position. Firms’ trajectories can provide the desired information about the strategy change’s effectiveness, as they reveal how the firm’s position changes relative to its former competitors (i.e., its previous position) and its new competitors (i.e., its targeted position).

Moreover, the firm needs to redefine its competitive set. As repositioning efforts of multiple firms can coincide, the firm may face both new and former competitors after successfully repositioning itself. Firms’ trajectories can help managers anticipate their future competitive situation by providing a forward-looking perspective on competition. Thereby, they can help inform corrective actions where necessary. For example, some of Meta’s (previously Facebook) strongest competitors (e.g., Google) are also positioning themselves in the virtual reality space (e.g., the metaverse). At the same time, firms from other parts of the market, such as gaming developers (e.g., Epic Games) and hardware manufacturers (e.g., Sony), are pushing into the realm of virtual reality. Although there is little that firms can do about other firms’ trajectories, these trajectories can alert them to emerging threats (i.e., unanticipated competition) and opportunities (e.g., potential collaborators) at an early stage.

This article aims to (1) provide a novel dynamic mapping framework that reveals firms’ trajectories and (2) illustrate the additional market structure insights that these trajectories create. Identifying firms’ trajectories is, however, not trivial. Although the marketing literature previously proposed several approaches for mapping changes in market structure, we find that these approaches are unfit to identify firms’ trajectories. Common to extant approaches is the idea of independently generating a sequence of static maps and connecting them so that changes in a market’s structure become apparent. By inspecting the change of firms’ positions from one static map to the next, marketers sought to learn how the market under analysis evolved. However, such a simple approach is limited in its capacity to accurately reveal market structure evolution because it (1) tends to generate a sequence of maps that are misaligned, (2) fails to uncover trends that persist over multiple periods, and (3) is sensitive to even low levels of noise in the data.

Specifically, a sequence of independently generated maps tends to be misaligned for two reasons. First, the coordinate systems of subsequent maps can change arbitrarily (Schönemann and Carroll 1970). Second, modern mapping methods typically do not have a unique solution but can approximate any given data in multiple ways. Thus, they can produce a sequence of substantially different maps when applied sequentially. Consequently, changes in the absolute positions of firms, or even in the relative distances between them, can be arbitrary. So, they do not necessarily correspond to actual changes in market structure. To mitigate the misalignment problem, marketers traditionally resort to Procrustes analysis (Gower 1975). Procrustes analysis tries to align multiple maps after their estimation by transforming their coordinate systems (e.g., rotations and reflections). Procrustes analysis can be an effective approach when the sequence of maps covers only a few periods and when one uses linear mapping methods, such as metric ratio multidimensional scaling (MDS) or centroid scaling (see, for instance, Moore and Winer 1987, Borg and Groenen 2005). However, as we later show, Procrustes analysis works less well with modern (nonlinear) mapping methods commonly used in contemporary market structure research (e.g., Ringel and Skiera 2016, Gabel et al. 2019, Yang et al. 2022) or when the analysis covers many periods.

Furthermore, changes in firms’ positions across multiple (more than two) successive maps can be erratic when one generates each map independently; that is, they can move back and forth instead of evolving continuously. Such erratic changes in map positions thwart any attempt to faithfully reveal persistent trends in firms’ competitive positions, that is, their trajectories.

Finally, because even tiny changes in the underlying relationship structure can affect each individual map, a sequence of maps is very sensitive to noise in the data. This sensitivity to noise exacerbates successive maps’ misalignment, curtails the revelation of trends, and may compromise the accurate representation of firms’ actual relationships on each map.

The limitations of previous approaches for mapping market structure evolution result from the fact that a map inherently simplifies a set of competitive relationships based on a particular set of criteria—none of which covers the alignment of firms’ positions across multiple maps or the identification of persistent trends over multiple periods.

Herein we propose a novel dynamic mapping framework called EvoMap, which overcomes previous approaches' limitations and is the first to accurately reveal firms’ trajectories in evolving market structures. EvoMap can accommodate a wide range of static mapping methods, such that one can easily apply it to various empirical settings and various kinds of data. The basic idea of EvoMap is to combine the estimation of multiple successive maps into a joint optimization problem. Thereby, we introduce additional objectives to achieve alignment, persistence, and noise cancelation. We then merge the estimated sequence of maps into a single dynamic market structure map that reveals each firm’s trajectory over time relative to all other firms’ trajectories.

In this article, we implement EvoMap for a classic psychometric method (metric MDS), a nonlinear variant (Sammon Mapping; Sammon 1969), as well as a more recent innovation in computer science (t-Distributed Stochastic Neighbor Embedding (t-SNE); Maaten and Hinton 2008). In an extensive simulation study, we validate that EvoMap accurately recovers market structure evolution across various possible market structures and different evolution patterns. Our simulation shows that EvoMap outperforms all previous approaches in its capacity to reveal accurate underlying trends in the changes in firms’ positions. Our findings are consistent across all three static mapping methods (i.e., metric MDS, Sammon Mapping, and t-SNE); that is, the superior performance of EvoMap is not contingent on a particular mapping method.

We use EvoMap (paired with t-SNE) to empirically investigate the evolution of product market competition among publicly listed firms over two decades. Our analysis uses the Text-Industry Network Classification (TNIC) data that Hoberg and Phillips (2016) provide. These data capture how firms position themselves relative to other firms based on product descriptions in their annual report to the U.S. Securities and Exchange Commission (SEC) (called Form 10-K). Using EvoMap, we identify the trajectories of 1,092 firms across 20 years and confirm the face validity of the revealed trajectories. Our analysis identifies individual firms’ repositioning efforts and more global changes in market structure, such as the convergence of entire industries.

The remainder of this article is structured as follows: The following section embeds this article into the extant literature on dynamic market structure mapping and details the limitations of previous approaches. We then formally introduce our proposed framework. Subsequently, we validate and benchmark it against alternative mapping approaches in an extensive simulation study. Finally, we use our framework to empirically investigate the evolution of product market competition based on the TNIC data. We close with a discussion on implications, limitations, and directions for future research.

2. Extant Approaches and Their Limitations

This article contributes to the literature on market structure analysis, specifically to the literature stream that seeks to capture complex networks of competitive relationships among entities of interest (e.g., firms, brands, or products) in spatial representations (maps). Without loss of generality, we limit the subsequent text to firms as the object of analysis.

2.1. Background on Mapping Market Structure

Studies that analyze market structure in maps usually adopt one of two approaches to represent the firms in a market: (1) as vectors in higher-dimensional space or (2) as nodes in a network (Wei 2020). The vector space approach models firms as vectors of d dimensions, where each of the d dimensions corresponds to one of the firms’ (potentially latent) attributes. One can then measure firms’ competitive relationships as pairwise dissimilarities or similarities between these vectors. Alternatively, the network approach models firms as nodes connected by edges that represent competitive relationships (measured as pairwise similarities). In both approaches, the basis for deriving a market structure map is a square matrix of pairwise competitive relationships (hereafter referred to as a competitive relationship matrix).¹ Typically, the matrix is symmetric (or symmetrized as part of the mapping process), but methods exist for asymmetric relationships (DeSarbo et al. 2006).

Given a competitive relationship matrix $\mathit{Y}\in {\mathrm{ℝ}}_0^{+}{}^{n\times n}$, static mapping methods fit a map—defined as a configuration of the set of firms I in lower-dimensional space $\mathit{X}=\left\{{\mathit{x}}_i=\left(x_i^{(1)},\dots , x_i^{(d_{\mathit{map}})}\right)\in {\mathrm{ℝ}}^{d_{\mathit{map}}}|i\in I\right\}$, $\mathit{X}\in {\mathrm{ℝ}}^{n\times d_{\mathit{map}}}, n=\left|I\right|$—to the competitive relationship matrix $\mathit{Y}$ as well as possible. The resultant configuration is usually two-dimensional to ease interpretation. Therefore, we also focus on two-dimensional maps (i.e., $d_{\mathit{map}}=2$). Yet, one- or three-dimensional applications also exist. Firms with strong competitive relationships appear close together on the map, whereas firms with weak competitive relationships appear more distant.

Commonly used mapping methods process either dissimilarity matrices (as in the case of MDS) or similarity matrices (as in the case of force-directed drawing; Fruchterman and Reingold 1991). Although some mapping methods use higher-dimensional vector data as their input, they eventually transform it internally into a relationship structure (e.g., by taking distances). Both classes of methods share similar underlying concepts, and each representation translates easily into the other (e.g., by taking the inverse). Therefore, we do not differentiate between similarities or dissimilarities but refer to weak competitive relationships (low similarities/high dissimilarities) or strong competitive relationships (high similarities/low dissimilarities).

Earlier studies of market structure largely measured competitive relationships from surveys (Sabnis and Grewal 2012). However, more recent marketing studies draw on a range of alternative data sources to measure competitive relationships. These new types of data—which include data from online search (Kim et al. 2011), online forum discussions (Netzer et al. 2012), customer reviews (Tirunillai and Tellis 2014, France and Ghose 2016), clickstreams (Ringel and Skiera 2016), cofollower networks in social media (Culotta and Cutler 2016), social tags (Nam et al. 2017), firm-related images (Liu et al. 2020), or user engagement in social media (Yang et al. 2022)—are often regularly retrievable over time in high frequency. Taken together, the abundance, accessibility, and high frequency of data on competitive relationships provide new opportunities to study market structure evolution and to create insights beyond those of single snapshots in time.

2.2. Marketing Approaches for Mapping Market Structure Evolution

Initial approaches in marketing for analyzing market structure in maps over time typically followed one of two approaches: (I) fixing the coordinate system ex ante or (II) aligning mapping solutions ex post.

2.2.1. Approach I: Fixing the Coordinate System Ex Ante.

The first approach fixes two dimensions as the map’s coordinate system. For example, one might decide to use the attributes performance and ease-of-use for the two dimensions of the coordinate system. One can then plot each firm’s position on these two attributes at each period to show how firms’ positions evolved. In other words, one describes the evolution of a market’s structure using a time-indexed scatter plot along two fixed dimensions. Examples from marketing research that apply this approach include van Heerde et al. (2004) and Tirunillai and Tellis (2014).

The limitation of this approach is that one needs to assume that the chosen two attributes sufficiently explain all relevant differences between firms over time. To relax this limitation, market researchers can resort to three-way MDS. Three-way MDS is an extension of MDS applicable whenever multiple relationship matrices for the same objects (e.g., firms) are available (for instance, one for each of several respondents in a survey). In the context of market structure analysis, three-way MDS can be used to estimate two latent dimensions and firms’ positions on them from multiple relationship matrices (Carroll and Chang 1970, DeSarbo and Carroll 1985, Cooper 1988, Mela et al. 1998). However, suppose these matrices represent successive periods. In such a case, three-way MDS assumes that the rank order of firms’ positions remains the same on each dimension (and merely the importance of each dimension changes over time). Moreover, the two dimensions themselves need to remain constant over time. Yet, these assumptions might be challenging to justify in more complex and dynamic markets, where maps can be particularly insightful.

2.2.2. Approach II: Aligning Mapping Solutions Ex Post.

For the second approach, one independently applies a static mapping method (e.g., MDS or t-SNE) at each period. The result is a sequence of individual maps that are not necessarily aligned (see panel (a) of Figure 1 for an example). One reason for such misalignment is that the maps are oriented differently (i.e., their coordinate system differs), as any mapping solution is unique only up to a linear transformation of its coordinate system (such as rotation or reflection).

To resolve the misalignment of independently generated maps, one can employ Procrustes analysis (e.g., Moore and Winer 1987), which identifies linear transformations (i.e., translation, rotation, reflection, or scaling) to align successive maps as well as possible (Schönemann 1966, Schönemann and Carroll 1970). Procrustes analysis tends to work well for methods that provide a linear mapping between the given relationship data and the proximities on the estimated map, such as metric ratio MDS (Borg and Groenen 2005). However, as shown in panel (b) of Figure 1, Procrustes analysis works less well for modern nonlinear mapping methods (e.g., t-SNE). Although the successive maps in panel (b) are more similar than in panel (a), substantial differences remain. In particular, the erratic shifts in cluster 1’s position remain even after applying Procrustes analysis.

One reason for such erratic shifts is that it is usually impossible to perfectly preserve an entire matrix of pairwise competitive relationships on a two-dimensional plane. In other words, a map is always an approximation of the underlying relationship matrix (Tversky and Hutchinson 1986). When the utilized mapping method’s objective function has multiple local optima, different maps approximate the same competitive relationship matrix equally well. Thus, repeatedly applying static mapping methods tends to result in a sequence of misaligned maps.

As Figure 1 shows, such misalignment is particularly prevalent for modern nonlinear mapping methods (e.g., t-SNE) typically used when the number of objects (i.e., firms) or clusters (i.e., submarkets) in a map is large—as is the case in many of today’s large markets (e.g., Ringel and Skiera 2016, Gabel et al. 2019). These nonlinear mapping methods are more flexible in how objects can be arranged on a map (for instance, by focusing more on placing objects with strong relationships nearby rather than objects with weak relationships far apart). However, this flexibility comes at the cost of allowing the method to rearrange weakly related objects in various ways without compromising the mapping quality (i.e., many local optima exist). Because these local optima are no longer unique up to linear transformations, the entailing sequence of maps cannot be aligned ex post via Procrustes analysis.

Some studies suggest resolving the local optima problem by various initialization strategies, such as using the same (i.e., fixed) random initialization in each period or initializing periods sequentially with the estimated map of the previous period (e.g., Gabel et al. 2019). However, because the estimated positions remain sensitive to even minor differences across successive competitive relationship matrices, these initialization strategies can easily fall short of producing well-aligned maps.

Moreover, even when using linear mapping methods (such as metric MDS), ex post alignment may not always lead to desirable outcomes. Specifically, by generating maps independently, one risks overfitting each map to even minor variations in the relationship data that may constitute noise. As Procrustes analysis only changes each map’s orientation, it cannot remedy such overfitting. The resultant overfitting to each period makes it hard to reveal persistent trends and can impair the accurate representation of the market’s structure. Moreover, Procrustes analysis does not tie the alignment over time to changes in the input data. It merely attempts to make maps similar over time without considering which firms changed their relationships with other firms and, thus, their positions (and to what extent). In sum, these extant approaches are subject to limitations that make them less suitable for many of today’s large, fragmented, and rapidly evolving markets.

2.3. Other Approaches for Mapping Evolving Relationship Data

The problem of mapping evolving relationships among objects is not unique to market structure analysis. Outside of the marketing literature, researchers proposed alternative approaches for visualizing changes in relationship data over time. For instance, in network visualization, Xu et al. (2013) propose increasing the stability of successive maps by preserving objects’ positions as much as possible over time through a regularization scheme. Rauber et al. (2016) later apply the idea of Xu et al. (2013) to t-SNE. They show that their “Dynamic t-SNE” increases objects’ stability in pairwise comparisons of subsequent snapshots in time relative to independently generated maps.

However, map stability alone is not sufficient when the objective is to study market structure evolution. Specifically, a very stable solution is undesirable when parts (or individual firms) of the market undergo substantial changes, whereas other parts (or individual firms) do not. Ideally, any firm’s map position should be as stable as possible across a sequence of maps as long as there are no substantial changes in that firm’s actual position (as indicated by its competitive relationships with other firms in the market). Moreover, Dynamic t-SNE merely links objects’ positions across two successive periods. Yet, as we will show in an extensive simulation study, doing so still falls short of revealing persistent trends.

2.4. Implications for Market Research

The limitations of extant approaches for mapping market structure evolution (i.e., misalignment, lack of persistence, sensitivity to noise) curtail market researchers’ abilities to study market structure evolution on multiple fronts:

First, when successive maps are misaligned, firms’ positions change erratically (e.g., a group of firms “jumping” to another place). As a result, it becomes impossible to identify firms’ trajectories and characterize their directions and lengths (e.g., to differentiate firms whose competitive relationships changed substantially versus those whose competitive relationships did not change).

Second, faithfully identifying underlying trends in firms’ positions (for instance, to extrapolate them into the future) is not easily possible when changes are not persistent but revert. Both sequences of maps in Figure 1 exhibit highly nonpersistent changes: The average correlation of successive changes in firms’ positions on these maps is −0.46 and −0.54, respectively, which indicates that any move of a firm into one direction of the map tends to be followed by a move into the opposite direction.

Third, the high sensitivity of individual maps to changes in the input data makes inference of actual market structure evolution harder. When, for instance, two firms move closer together (further apart) on a map, it is unclear whether this convergence (divergence) is due to (1) actual changes in the firms’ positions, (2) noise in the data, or (3) different local optima (or a combination thereof). This problem remains for approaches that constrain the movement of all firms uniformly, as these approaches risk constraining the movement of some firms more (or less) than what is justified by the underlying data.

We overcome these limitations of past approaches with a novel dynamic mapping framework called EvoMap. In contrast to past approaches in the marketing literature, EvoMap jointly estimates a sequence of maps from a sequence of competitive relationship matrices instead of attempting to align maps ex post. EvoMap thereby not only ensures stability in successive maps as proposed by Xu et al. (2013) but explicitly accounts for heterogeneity in how strongly firms change their positions.

To this end, EvoMap uses a novel adaptive regularization scheme, which adjusts constraints for each firm based on the data at hand. To reveal persistent trends in the evolution of firms’ positions, EvoMap additionally smooths the estimated trajectories across multiple periods by imposing further constraints on them. As a result, the estimated trajectories do not overfit individual periods (which would result in erratic oscillation) but reveal smooth underlying trends that ease map exploration and allow to extrapolate into the future. This “noise cancellation” property of EvoMap makes it particularly suitable for high-frequency data sources that are likely subject to a substantial amount of noise, as in the case of, for example, relationships derived from user-generated content or social media. Finally, in contrast to recent approaches (i.e., Rauber et al. 2016), we design EvoMap as a flexible framework that can easily accommodate various static mapping methods (e.g., MDS, Sammon Mapping, t-SNE, or methods not yet developed).

Table 1 summarizes the key differences between EvoMap and extant approaches for mapping market structure evolution.

Table 1. Alternative Approaches for Mapping Market Structure Evolution

3. Formal Description of EvoMap

Let $\mathit{Y}\in {\mathrm{ℝ}}_0^{+}{}^{n\times n}$ denote a competitive relationship matrix representing the market structure for the set of firms $I$ with $n=\left|I\right|$. The entries of $\mathit{Y}$ consist of symmetric measures of pairwise competitive relationships. We do not impose any restrictions on how these measures are derived. That is, they could represent measures of substitutability, brand switching, co-occurrence, or distances in a high-dimensional embedding space (among many others). We also do not restrict the type of relationships (i.e., distances/dissimilarities versus similarities), which depends upon the chosen static mapping method.

Static mapping methods estimate the map $\widehat{\mathit{X}}\in {\mathrm{ℝ}}^{n\times 2}$ for any given competitive relationship matrix $\mathit{Y}$ by minimizing the cost function $C\left(\mathit{X}\right): {\mathrm{ℝ}}^{n\times 2}\to {\mathrm{ℝ}}_0^{+}$, which measures the discrepancy between firms’ competitive relationships in the matrix $\mathit{Y}$ and the relative proximities of their positions on a map $\mathit{X}$:

Note that different cost functions $C$ give rise to different mapping methods. We assume that C is a nonnegative loss function such that smaller values correspond to better solutions. For instance, C could represent Kruskal’s stress function (for MDS) or Kullback-Leibler divergence (for t-SNE). If solution quality increases with increasing values of C or if C takes negative values (e.g., for a log-likelihood function), one first needs to transform C to use it within our framework (e.g., invert it).

Applied to a sequence of competitive relationships matrices ${\left({\mathit{Y}}_t\right)}_{t=1,\dots ,T}$, static mapping methods derive a sequence of maps ${({\widehat{\mathit{X}}}_t)}_{t=1,\dots ,T}$ such that each map ${\widehat{\mathit{X}}}_t$ at time t preserves the competitive relationships in ${\mathit{Y}}_t$ as well as possible. In addition, we pursue the following three objectives:

1. Successive maps should be aligned. That is, the total changes in positions on successive maps $\Vert {\widehat{\mathit{X}}}_t-{\widehat{\mathit{X}}}_{t-1}\Vert ,t=2,\dots , T$ should be small.

2. Changes in firms’ positions should persist across successive periods; that is, their trajectories should reveal gradual underlying trends rather than oscillate back and forth. Specifically, successive movements ${(\nabla {\widehat{\mathit{X}}}_t)}_{t=2, \dots , T }$, where $\nabla {\widehat{\mathit{X}}}_t={\widehat{\mathit{X}}}_t-{\widehat{\mathit{X}}}_{t-1}$, should exhibit nonnegative serial correlation.

3. Changes in firms’ positions on the map should reflect actual changes in market structure. That is, any firm’s trajectory length on the map should be related to the degree of change in its actual position (as indicated by its competitive relationships).

Meeting one objective might come at the cost of another. Consider a sequence of relationship matrices subject to some degree of change. An utterly stable sequence of maps ${\widehat{\mathit{X}}}_1={\widehat{\mathit{X}}}_2=\dots ={\widehat{\mathit{X}}}_T$, for instance, aligns successive maps perfectly but reduces the goodness-of-fit of each map. Moreover, the absence of changes on the map does not reflect the existing changes in the competitive relationship matrices. Thus, we cannot expect to meet all objectives perfectly. We, therefore, balance these objectives in the following joint optimization problem.

3.1. Optimization Problem for EvoMap

The basis for EvoMap is a static mapping method, such as MDS, Sammon Mapping, or t-SNE. We design EvoMap as a flexible framework to accommodate various static mapping methods. This flexibility is important for marketing research because it allows researchers to select the mapping method that is most suitable for their specific purpose, empirical setting, and data source. Notably, EvoMap’s flexibility opens it to future advances in static mapping methods.

To meet the three additional mapping objectives outlined in the previous subsection, we proceed as follows: Rather than fitting each map independently to the respective competitive relationship matrix for all $t\in \left\{1,\dots T\right\}$, we fit the sequence of maps jointly to the sequence of competitive relationship matrices. Doing so allows us to (1) incorporate information about maps in successive periods into the estimation of each map and to (2) add additional objectives.

Formally, we derive the sequence of maps ${({\widehat{\mathit{X}}}_t)}_{t=1,\dots ,T}$ as

In Equation (3), the first term represents the static component of EvoMap’s cost function, which equals the sum of the cost function $C_{\mathit{static}}$ of the selected static mapping method evaluated at each period $t\in \left\{1,\dots ,T\right\}$. This static component of EvoMap’s cost function seeks to preserve the competitive relationships in ${\mathit{Y}}_t$ on the map ${\mathit{X}}_t$ as closely as possible for every $t\in \left\{1,\dots ,T\right\}$. The second term represents the temporal component of the cost function, which seeks to meet the three additional objectives outlined before. The hyperparameter $\alpha \in {\mathrm{ℝ}}_0^{+}$ balances the relative importance of the cost functions’ static versus temporal component. We specify $C_{\mathit{temporal}}$ as follows:

$1_{\left[i\in I_{t,k}\right]}: I\to \left\{0,1\right\}$ denotes an indicator function equal to one if firm i is present in time $t$ and the $k$ preceding periods $1_{\left[i\in I_{t,k}\right]}≔{\prod }_{l=0}^k{1}_{\left[i\in I_{t-l}\right]}$, where $1_{\left[i\in I_t\right]}≔\{\begin{array}{c}1\\ 0 \end{array} if \begin{array}{c}i\in I_t\\ i\notin I_t\end{array}$,

$f_w:I\to {\mathrm{ℝ}}^{+}$ is a positive weight function defined on the set of firms I,

$p\in \mathrm{\mathbb{N}}$ is a second hyperparameter of positive integer values that controls the degree of smoothing, and

${\nabla }^k{\mathit{x}}_{i,t}\in {\mathrm{ℝ}}^2$ is the k-th (backward) difference of firm $i$’s map position at time $t$:

For $k=0$, ${\nabla }^k{\mathit{x}}_{i,t}$ simply is firm i’s position at time t:

For $k=1$, ${\nabla }^k{\mathit{x}}_{i,t}$ equals the difference vector between firm $i$’s positions in $t$ and $t-1$:

For $k>1$, ${\nabla }^k{\mathit{x}}_{i,t}$ represents a higher-order difference vector (e.g., for $k=2$, it corresponds to the difference in differences of successive positions, analogous to the acceleration). We outline the underlying intuition of the terms in Equation (4) in what follows. We also provide an illustrative example of its calculation together with exemplary trajectories in Online Appendix B.

First, to align successive maps, we link firms’ positions in successive periods in our cost function. Specifically, we penalize all changes between successive maps, measured as the squared Euclidean norm ${\Vert {\nabla }^1{\mathit{x}}_{i,t}\Vert }^2$ of the first differences between each firm’s positions in $t$ and $t-1$, summed over all firms.² Doing so penalizes large changes in firms’ positions over time and thereby aligns successive maps.

To achieve our second aim (identifying trajectories that capture underlying trends), the penalty term ${\Vert {\nabla }^1{\mathit{x}}_{i,t}\Vert }^2$ alone is insufficient as merely two successive periods impact each firm’s position in a given period. As a result, ${\Vert {\nabla }^1{\mathit{x}}_{i,t}\Vert }^2$ would, for instance, penalize oscillation of firms’ positions equally strong as a smooth trajectory into one direction, as long as the total length of the movement path is the same (for a concrete example, see Online Appendix B). Therefore, we extend the penalty term such that more than two successive periods impact each firm’s position in a given period. To do so, we penalize higher-order differences between any firm’s successive positions ${\Vert {\nabla }^k{\mathit{x}}_{i,t}\Vert }^2$. For $k=2$, for instance, this term penalizes large differences in differences (analogous to the acceleration of a firm’s moves on the map). Effectively, these penalty terms impose stronger constraints on the resultant trajectories by penalizing more complex trajectories (for instance, erratic back-and-forth movement). As a result, the estimated trajectories become less sensitive to changes in individual periods and better recover underlying trends across multiple periods. Hyperparameter $p$ controls the degree of smoothing.

To achieve our third aim (i.e., to link changes on the map to actual changes in market structure), we further augment the cost function with the weighting function $f_w$. Until now, the outlined specification imposes constraints on all firms’ trajectories uniformly—regardless of how much their competitive relationships with others changed. Doing so, however, harms the ability to differentiate between firms with dynamic and static positions. Specifically, uniform penalties across all firms are undesirable because firms may change their competitive relationships with other firms at different rates and to different extents. Consider, for instance, a new entrant heading toward the market’s incumbents (i.e., its competitive relationships with all incumbents are strengthening). Overall, the entrant’s competitive relationships are subject to substantial changes, whereas any incumbent’s competitive relationships may change only marginally (only the competitive relationship with the new entrant changes substantially). Ideally, the resultant sequence of maps should capture such heterogeneity by retaining incumbents at their current positions while allowing the new entrant to change its position toward the incumbents (such that its trajectory points toward them).

The weighting function $f_{\text{w}}:I\to {\mathrm{ℝ}}^{+}$ accounts for such heterogeneity across firms by determining how each firm $i$ is affected by the previously outlined constraints. Specifically, firms whose competitive relationships changed very little should be constrained from undergoing large changes in their positions on the map (i.e., they should receive high weights). In contrast, firms with substantial changes in their competitive relationships should be free to change their positions on the map more (i.e., they should receive low weights). Thus, we specify $f_{\text{w}}$ as a monotonically decreasing function of the total change in firm i’s competitive relationships. Specifically,

Here, ${\mathit{Y}}_{i,t}$ denotes the i-th row of the relationship matrix ${\mathit{Y}}_t$. Thus, ${\Vert {\mathit{Y}}_{i,t}-{\mathit{Y}}_{i,t-1}\Vert }^2$ captures the total change in firm i’s relationships with all other firms between periods $t-1$ and $t$; $z_i$ captures the total change in firm i’s relationships to all other firms across all periods. Note that the scale of $z_i$ depends on the scale of the input data, that is, the entries in $\mathit{Y}$ (for instance, pairwise similarities are typically bound by [0, 1], whereas pairwise distances can be much larger than one). To account for such potential differences in the input data’s scale, we use the normalizing constant $b$ in Equation (8).

Lastly, we add an indicator function $1_{\left[i\in I_{t,k}\right]}$ to include cases where not all firms are observable for the entire observation period (e.g., due to market entry or exit).

Combining Equations (3) and (4), we fit the sequence of maps according to (2), where

3.2. Python Implementation of EvoMap

Different mapping methods can be superior in different empirical contexts and for different tasks (France and Akkucuk 2021). Therefore, we designed EvoMap independent of a particular static mapping method (the choice of which only determines the specification of $C_{\mathit{static}}$, its corresponding component in the gradient, and potentially its optimization technique). As outlined before, this flexibility allows market researchers to decide which static mapping method is most suitable for their specific empirical setting.

In this article, we implement EvoMap for three popular mapping methods to demonstrate that EvoMap is compatible with (I) traditional psychometric methods, (II) their nonlinear advancements, and (III) recent innovations in computer science:

1. metric MDS,

2. Sammon Mapping (Sammon 1969), and

3. t-SNE (Maaten and Hinton 2008).

We implement EvoMap in Python, adopt each mapping method's respective static cost function, and derive the cost-minimizing solutions via iterative optimization with adaptive step sizes. Specifically, we use gradient descent with backtracking (via step halving) for metric MDS and Sammon Mapping. For t-SNE, we use momentum-based gradient descent with early exaggeration. Further, we automatically adjust initial step sizes via exponential decay to ensure convergence. We provide the gradient derivations in Online Appendix A.

Our Python implementation draws on the NumPy library (https://numpy.org) for numerical computing and the Numba compiler (https://numba.pydata.org) for performance optimization. Our implementation is available as a Python package with this journal article; see https://github.com/mpmatthe/evomap. We provide further details on EvoMap’s implementation (including the specific package versions and runtime estimates) in Online Appendix E.

3.3. Hyperparameter Selection

The hyperparameters $\text{α}$ and $p$ give market researchers additional control over their mapping solutions. To illustrate and quantify the hyperparameters’ effects on the resulting sequence of maps, we use the following three metrics:

1. Static goodness-of-fit is measured as the average Hit-Rate of k-nearest neighbor recovery across all T periods and n firms, adjusted for random agreement: $\frac{1}{T} {\sum }_{t=1}^{T}A{R}_k^{*}$, where $A{R}_k^{*}=\frac{1}{nk}{\sum }_{i=1}^n{a}_{ik}-\frac{k}{n-1}$ and $a_{ik}$ denotes the number of firm i’s shared k-nearest neighbors in the data and on the map (France and Carroll 2007, Chen and Buja 2009). The choice of k depends upon the application and can be adjusted to obtain a measure of local recovery (when k is small) or global recovery (when k is large). Given that market structure analysis typically focuses on identifying close competitors (rather than remote ones), we follow Ringel and Skiera (2016) and set k to 10.

2. Misalignment is measured as the average movement path length across successive positions: $\frac{1}{(T-1)}{\sum }_{t=2}^T\frac{1}{n}{\sum }_{i=1}^n\Vert {\mathit{x}}_{i,t}-{\mathit{x}}_{i,t-1}\Vert$, using the Euclidean norm.

3. Persistence is measured as the Pearson correlation coefficient between successive changes in all firms’ positions (averaged across both dimensions of the map).

Figure 2(a) shows that when $\alpha$ is set very low, the static goodness-of-fit of each map (i.e., the average Hit-Rate) is highest. Yet, maps tend to be misaligned (panel (b)), and the resultant trajectories on successive maps tend to evolve erratically (panel (c)). As $\alpha$ increases, misalignment decreases (panel (b)) and the trajectories’ persistence tends to increase (panel (c)), while—at some point—the static goodness-of-fit of each map decreases considerably (panel (a)).

The second hyperparameter $p$ thereby defines the highest order of differences considered in the penalty term. As such, $p$ effectively sets the degree of smoothing by controlling how many successive periods impact each firm’s map position at a given period. For $p=1$, the penalty only considers first-order differences (corresponding to two successive periods). For $p=2$, the penalty also considers second-order differences (corresponding to three interdependent periods) and so on (i.e., $p+1$ periods). A higher value of $p$ sets higher constraints, producing smoother trajectories that are less sensitive to changes in individual periods.

As evident from Figure 2, higher values of $p$ make it cheaper (in terms of static goodness-of-fit) to achieve more persistent trajectories (panels (a) and (c)). Conditional on a specific value of $\alpha$, persistence is substantially higher for increasing values of $p$ (see panel (c) of Figure 2), whereas the corresponding decrease in static goodness-of-fit is small (see panel (a) of Figure 2). Hence, one can choose to trade off small decreases in static goodness-of-fit against much smoother trajectories by imposing higher constraints (i.e., setting $p$ to a higher value).

Based on the observed trade-offs, we propose the following strategy for tuning the hyperparameters $\alpha$ and $p$. First, we select a threshold for how much static goodness-of-fit we are willing to “trade” for better dynamic mapping quality. For instance, we might set this threshold to 5% in terms of the average Hit-Rate. Then, starting with low$\alpha$ values (e.g., $\text{α}$ near zero), we increase $\text{α}$ until the static mapping quality drops below this threshold. In consequence, the entailing sequence of maps will align as strongly as possible without impairing the static mapping quality beyond the threshold. We repeat the procedure for multiple values of $p$ (starting with $p=1$) to obtain a set of candidate value pairs ($\alpha$, $p$) of which we pick the one that produces the visually most appealing solution. Importantly, EvoMap is an unsupervised learning framework that is explorative. Therefore, there is no straightforward procedure for finding an optimal hyperparameter combination for each specific data set (such as cross-validation in supervised learning). We, therefore, recommend that analysts always test multiple hyperparameter combinations and fine-tune them via visual inspection.

Although Figure 2 provides empirical evidence for EvoMap’s ability to produce sequences of well-aligned maps with persistent firm trajectories, it is confined to a single application in which the ground truth is unknown. To obtain a more comprehensive and rigorous understanding of EvoMap’s capabilities, we proceed with an extensive simulation study in the next section.

4. Simulation Study

The objectives of our simulation study are fourfold: (1) to validate EvoMap by testing its ability to accurately recover various simulated market structures’ evolution; (2) to compare EvoMap with alternative dynamic mapping approaches; (3) to study EvoMap’s ability to distinguish underlying trends from noise in the data (i.e., avoid overfitting); and (4) to test EvoMap’s suitability as a flexible framework that can accommodate various static mapping methods.

Our simulation study comprises four steps (see Figure 3 for a schematic overview). In the first step, we simulate firms’ positions as vectors in a higher-dimensional attribute space (number of dimensions $d>2$) with preimposed cluster structures representing different submarkets. Next, we gradually adjust firms’ simulated positions according to three evolution scenarios: (1) emergence of a new submarket, (2) shifts in individual firm’s positions, and (3) market entry of new competitors. We design these three evolution scenarios to capture fundamental patterns that we expect to observe in the evolution of market structures. Our three proposed evolution scenarios’ primary drivers are new consumer needs, firms’ repositioning and differentiation efforts, innovations, and new market entries. Finally, we add a varying degree of Gaussian white noise to all simulated positions to make our simulations more realistic.

In the second step, we derive a competitive relationship matrix for each period by measuring the pairwise squared Euclidean distances among all (noisy) simulated positions in the higher-dimensional attribute space. As a result, we obtain a time-indexed sequence of competitive relationship matrices representing the simulated market structure’s evolution and some degree of noise (where each sequence corresponds to one of the three evolution scenarios). In the third step, we apply EvoMap and various alternatives to these sequences of competitive relationship matrices to estimate two-dimensional representations, that is, sequences of maps. Notably, we use noisy measurements for estimation (i.e., the relationships measured after adding noise to the simulated positions). In the fourth and final step, we evaluate each sequence of maps with regard to the desirable properties of dynamic market structure maps, namely, (1) successive maps should be aligned, (2) changes in firms’ positions should persist over successive periods, and (3) changes in firms’ positions should reflect actual changes in market structure. Note that we evaluate the resultant maps (estimated based on noisy measurements) against the actual market structure (that is, the simulated positions before adding any noise).

Appendix A provides a detailed description of our data-generating process. To illustrate the evolution scenarios, we generate six evolving market structures and map them with EvoMap in Figure 4.

4.1. Evaluation Criteria and Comparison

To evaluate and compare dynamic mapping approaches, we use the three metrics introduced in Section 3.3: Hit-Rate (HIT-RATE), Misalignment (MIS-ALIGN), and Persistence (PERS). To further evaluate how well the estimated trajectories correspond to actual changes in market structure, we introduce a fourth metric: Change-Correlation (C-CORR), the Pearson correlation coefficient between the movement path lengths of firms’ simulated and estimated positions, respectively. The intuition behind C-CORR is that firms should change their positions on a map in relation to how strongly their actual positions have changed.

HIT-RATE and C-CORR represent goodness-of-fit measures, which capture how well the estimated positions and trajectories fit the simulated data. MIS-ALIGN and PERS represent descriptive measures, which qualify different aspects of the estimated maps and trajectories. Naturally, alternative specifications for these metrics are possible, which we test in Online Appendix C.

Our simulation study comprises the following six dynamic mapping approaches: (1) EvoMap, (2) Dynamic t-SNE,³ (3) ex post alignment via Procrustes analysis, (4) sequential initialization of t-SNE, (5) fixed initialization of t-SNE, and (6) independent mapping (the benchmark for static mapping quality). Here, we focus on t-SNE as the static mapping method as (I) we also use it later in our empirical application and (II) it is frequently used in recent marketing studies (e.g., Gabel et al. 2019, Yang et al. 2022). Appendix B also evaluates EvoMap paired with additional mapping methods (i.e., stress-based metric MDS and Sammon Mapping).

4.2. Simulation Results

Visual inspection (as shown in Figure 4) confirms that EvoMap generates well-aligned maps (i.e., the coarse-grained configurations remain stable) in which individual positions still change over time. Firms do not erratically jump around in the estimated sequence of maps. Instead, firms tend to follow relatively gradual movement paths. Firms unaffected by the induced evolution scenarios are visually separable; they retain relatively stable positions. As a result, we can easily discern firms’ trajectories (marked as shaded movement paths in Figure 4).

Table 2 reports the results of our comparison, organized by the three market evolution scenarios using the previously outlined metrics: Hit-Rate, Misalignment, Persistence, and Change-Correlation. We report all metrics as means (standard deviation in parentheses) across 729 combinations for each of the three evolution scenarios. Thus, in total, we evaluate each approach on 2,187 simulated evolving market structures.

Table 2. Results of Simulation Study

The results show that EvoMap consistently outperforms alternative approaches across all metrics and in all three evolution scenarios by a substantial margin. In contrast to all alternatives, EvoMap estimates well-aligned maps (MIS-ALIGN is lowest), featuring gradually evolving trajectories (PERS is highest), which correspond well to the actual (i.e., simulated) changes in market structure (C-CORR is highest). Notably, static mapping quality is also higher than for all remaining approaches. To understand why, recall that we evaluate static mapping quality against the actual (simulated) positions (i.e., without any noise) but estimate maps using the noisy relationship matrices (i.e., the relationships measured after adding noise to the simulated positions). Thus, any approach that merely considers data from a single period cannot separate the actual positions from the additional noise in each period. EvoMap, in contrast, considers information from multiple successive periods when estimating the positions in each given period. Thereby, we can (partially) cancel-out noise in each period. As a result, the static mapping quality increases (HIT-RATE is highest).

Considering the alternative approaches in more detail, we make the following observations: our results confirm our earlier hypothesis that ex post alignment via Procrustes analysis performs poorly when applied to maps generated by nonlinear methods like t-SNE. Additional analyses (presented in Appendix B) show that Procrustes analysis performs better when paired with MDS—yet, still falls short of EvoMap (with MDS). In regard to the different initialization strategies, we find that sequential initialization performs best (specifically, MIS-ALIGN is low, indicating rather well-aligned maps). However, sequential initialization still exhibits strong sensitivity to minor changes (as indicated by low Persistence). Moreover, it does not recover actual changes in market structure well (C-CORR is low).

Finally, we find that Dynamic t-SNE achieves moderate Persistence and Change Correlation values but cannot consistently create well-aligned maps with good static mapping quality (indicated by high Misalignment and low Hit-Rates). Moreover, Dynamic t-SNE’s results are sensitive to the different types of market structure and often reach only poor local optima (indicated by low Hit-Rates and high standard deviations across all metrics). Thus, in Online Appendix C, we test whether we can improve Dynamic t-SNE’s performance using EvoMap’s optimization procedure. We find that doing so improves its performance, but results are still below the performance of EvoMap by a substantial margin.

4.3. Additional Analyses

EvoMap introduces several novelties. We conduct additional analyses with our simulated data to better understand the benefits of these novelties.

First, we investigate the benefits of EvoMap over alternative approaches when paired with other static mapping methods (i.e., MDS and Sammon Mapping). We find that EvoMap also produces superior results (see Appendix B for details).

Second, we disentangle the impact of adaptive regularization and smoothing on map quality. Our analysis reveals that both contribute individually to map quality, but the combination of both yields the highest goodness-of-fit (see Online Appendix C for details).

Third, we investigate EvoMap’s noise cancellation property more closely using hold-out validation. Specifically, we test the extent to which EvoMap avoids overfitting individual maps to noise in the data by holding out particular periods during the estimation and predicting their positions based on the estimated trajectories. In contrast to alternative approaches, EvoMap substantially mitigates the risk of overfitting (see Appendix C).

Finally, we test the sensitivity of our findings toward market structure characteristics in Online Appendix D. Regression analysis shows that the reported results do not suffer from pronounced sensitivity to specific market structure characteristics. Thus, EvoMap is well applicable for a wide range of market structures.

5. Empirical Study

In what follows, we aim to empirically show that EvoMap creates additional market structure insights beyond traditional (i.e., static) mapping methods. Specifically, we seek to (1) reveal firms’ trajectories using EvoMap, (2) test the estimated trajectories for face validity, (3) and illustrate the additional insights that firms’ trajectories provide. To do so, we use EvoMap to empirically investigate the evolution of publicly listed firms’ positions over two decades.

Specifically, we apply EvoMap to the Text-based Network Industry Classification data by Hoberg and Phillips (2016). These data consist of a time-indexed sequence of pairwise similarity measures among all publicly listed firms in the United States. Hoberg and Phillips (2016) derive these measures of pairwise firm similarity based on firms’ product descriptions in their 10-k SEC filings.

We chose these data for four reasons: First, these data span more than 20 years. Such an extended period provides the ideal empirical setting to assess EvoMap’s ability to identify trajectories that reveal persistent underlying changes in firms’ positions. Second, TNIC data are well established and validated in the finance and economics literature. Previous research leveraged these data to, for instance, study mergers and acquisitions, firms’ reactions to product market threats, the relationship between competition and the cost of capital, or innovation strategies in the information technology (IT) industry (Hoberg and Phillips 2010, Valta 2012, Hoberg et al. 2014, Kim et al. 2016, Li and Zhan 2019). Third, TNIC data feature well-known publicly listed firms, which facilitates assessing the face validity of the identified market structure and its evolution. Lastly, TNIC data provide sufficient complexity in market structure (i.e., many competitors, many submarkets) and include many sources of market structure evolution (e.g., changes in product portfolios, product innovations, or technological change).

5.1. Sample Construction

We retrieved the TNIC data from Hoberg and Phillips’ (2016) website: https://hobergphillips.tuck.dartmouth.edu/. The data comprise a time-indexed list of pairwise similarity measures, which we transform into a sequence of similarity matrices. Our observation period ranges from 1998–2017, with one observation per year, totaling 20 observations for each firm pair. The set of firms changes every year because of entries (i.e., initial public offerings) and exits (such as defaults, delisting, or acquisitions). To study firms’ trajectories across an extended period, we focus our analysis on firms present in each period. We prune the firm network to only those firms who (I) have at least three product market competitors in the sample in each year and (II) whose total similarity to other firms in the sample exceeds 1% in each year. Setting such a minimum threshold is required. Otherwise, our sample would include firms that barely relate to any other firm in our sample, such that we could not faithfully estimate their positions. Our final sample comprises 1,092 firms across 20 years. We match our sample to further data from Compustat (via the “gvkey” identifier) to obtain descriptive variables, such as the firm names and Standard Industrial Classification (SIC) codes.

5.2. Dynamic Mapping of TNIC Data

We apply EvoMap (with t-SNE) to our full sample; That is, to the pairwise relationships among all 1,092 firms for all 20 years. We set hyperparameters $\alpha$ and $p$ using our hyperparameter tuning procedure that we proposed in Section 3.3: We quantitatively evaluate a fine grid of 150 potential combinations (3 values for $p$ and 50 values for $\alpha$, as depicted in Figure 2); identify candidate values that trade-in approximately 5% of goodness-in-fit (in terms of the adjusted 10-nearest neighbor (NN) Hit-Rate) relative to independent mapping; and select the final hyperparameters based on visual evaluation of the candidate values’ results. Specifically, we set $\alpha =6.92\times {10}^{-4}$ and $p=3$.

Figure 5 shows the first and the last map from EvoMap’s estimated map sequence (i.e., 1998 and 2017). Both maps in Figure 5 reveal various submarkets (as indicated by the pronounced clustering of the bubbles in the maps) that are well aligned. To better understand the visualized submarket structures and to test for face validity, we manually label each submarket based on the firms included in each submarket. For instance, the technology submarket (bottom left) includes firms such as Apple, Microsoft, or Western Digital, whereas the financials submarket (top right) includes firms such as Capital One or Wells Fargo. All submarkets have an easily identifiable theme. Moreover, the relative positioning is intuitive (for instance, financials, investment, and insurance are positioned close together in the top right corner).

The comparison of two aligned static market structure maps, as presented in Figure 5, offers some insight into the evolution of market structure over time (e.g., changes in the submarket structure). However, it does not readily reveal how individual firms changed their positions. A dynamic market structure map, on the other hand, can reveal such changes in market structure based on firms’ trajectories. We present EvoMap’s dynamic market structure map across 20 years in Figure 6.

Figure 6 (panel (a)) reveals all firms’ trajectories from 1998 until 2017 as smooth, shaded paths. We find that the global market structure (i.e., relative positions of submarkets) remains stable and individual maps are well aligned. Although most firms change their position only to a small extent, some firms substantially change their competitive positions (as indicated by several long trajectories). Moreover, although the global submarket structure remains relatively stable, there is considerable movement of firms within and across submarkets. For instance, within the bottom-left part of the map (highlighted in panel (b) of Figure 6), technology firms (such as Apple) move closer to communication firms (such as AT&T), reflecting the increasingly important role of communication technologies for these firms’ products.

5.3. Evaluation

In what follows, we evaluate EvoMap’s trajectories with respect to (1) their face validity and (2) the substantive insights they can create.

5.3.1. Face Validity.

Figure 6 suggests that most firms do not change their positions relative to other firms in the market much (i.e., most trajectories are relatively short). However, several firms substantially changed their positions (as indicated by longer trajectories). Our observation confirms the intuition that there is heterogeneity regarding the extent to which firms change their competitive position over time. Although greater changes are easier to see in Figure 6, most firms adjust their position to some extent (that is, we observe almost no zero-length trajectories).

To test the face validity of the estimated trajectories, we average their length on the SIC code level. Intuitively, if the identified trajectories accurately uncover changes in firms’ positions, firms in less dynamic markets should (on average) have shorter trajectories than firms in more dynamic markets. To evaluate whether this relationship holds, we aggregate the average trajectory length by market membership (measured as the two-digit SIC code). Table 3 shows the 10 SIC codes with the longest and shortest average trajectory (measured as their movement path’s total length in Figure 6). Our findings are intuitive because we observe the shortest average trajectories in markets with rather stable product portfolios (e.g., printing, food stores, apparel). That is, firms in these markets, including the New York Times or Abercrombie and Fitch, did not exhibit substantial changes in the products they sell. For instance, although fashion trends evolve, fashion products (i.e., apparel) change little. Likewise, although the content published by news publishers changes constantly, their products themselves remain rather similar. For instance, the New York Times still sells news. In contrast, we observe the longest average trajectories in markets with rather dynamic product portfolios (e.g., retail, business services, trade, or computer equipment). Firms in these markets, including Adobe or Intuit, change their products relative to their peers much more substantially. Many software firms, for instance, have been shifting their solutions to cloud-based subscription models, whereas wholesale trade firms, such as United Natural Food, constantly need to adjust their product portfolios in response to changing trends (e.g., the emergence of meat substitutes, vegan foods, or sustainable products).

Table 3. Average Trajectory Length per SIC Code

5.3.2. Substantive Insights.

EvoMap’s trajectories offer additional substantive insights beyond static market structure maps. For example, monitoring an individual firm’s trajectory can reveal whether and how its position has changed in the past. Panel (a) of Figure 7 displays the trajectory of Capital One (a large financial service provider). Its trajectory reveals that Capital One has indeed changed its position substantially over the past 20 years. Moreover, we observe a pronounced bend. This sudden change of direction occurred around 2004 and coincided with the expansion of Capital One’s business model. In the early 1990s, Capital One solely focused on selling credit cards. Then, throughout the 1990s, Capital One expanded into auto loans. Finally, around 2005, Capital One expanded into consumer banking.

Likewise, the trajectory of American Express (panel (b) of Figure 7) reveals its departure from the investment submarket, where investment banks, such as Morgan Stanley, Oppenheimer, or Jefferies, are positioned (and remain at their position), toward the financials submarket, including retail banks. This observation coincides with the divesting of American Express’s investment banking units, which started in the early 1990s. The aforementioned trajectories are thereby insightful to the respective firms themselves and other firms in the market because these trajectories may alert them to emerging competitors at an early stage.

Besides studying each firm’s trajectory, market researchers can gain additional insights from studying firms’ trajectories relative to each other. Panel (c) shows how the trajectory of Walmart diverges from grocery stores (such as Weis Markets or United Natural Foods) and starts to move closer toward apparel firms (such as Urban Outfitters or L Brands), revealing the growing expansion of its fashion assortments featuring both national and private brands.

Likewise, panel (d) of Figure 7 shows the trajectories for Apple versus some of its peers (such as Western Digital). The map shows that Apple and Western Digital were positioned closely in 1998. Both firms provided computer hardware, together with other hardware providers, such as HP. Over time, however, Apple’s and Western Digital’s trajectories diverged. Although the trajectory of Western Digital converges with other hardware providers, such as Micron, the trajectory of Apple points in a different direction. Eventually, Apple’s trajectory converges with digital services providers, such as Intuit. The recovered trajectory for Apple reflects Apple’s evolving business model, which evolved from mainly selling hardware (such as the Mac) toward software and digital services (such as its AppStore or iTunes). Although a static market structure map (for instance, around 2010) would place the three firms at approximately equal distance, EvoMap’s dynamic market structure map indicates the divergence of Apple’s and Western Digital’s positions. Indeed, a few years later, Apple and Western Digital faced very different competitors.

The above examples highlight the value of mapping firms’ trajectories because they reveal firms’ (re)positioning efforts at an individual level and relative to one another. We make our empirical study’s full results available to readers in an interactive browser-based online tool specifically developed for this article (visit www.evomap.io/). Using our tool, readers can easily explore the trajectories of various firms of interest on their own.

Finally, we can also use EvoMap to obtain a fine-grained picture of a smaller set of firms in the market (using, for instance, EvoMap implemented for metric MDS rather than t-SNE). We provide the results for such an analysis of 10 competitors within the technology submarket in Online Appendix F. The results reflect similar evolution patterns, supporting our findings' robustness.

6. Discussion and Conclusion

This article presents a novel dynamic mapping framework called EvoMap that uses longitudinal data on firms’ competitive relationships to reveal firms’ trajectories within evolving market structures. Prior marketing research highlights the value of, and need for, monitoring the evolution of market structures. Although high-frequency, longitudinal data on competitive relationships among firms are increasingly available, market structure analysis has not yet fully unlocked the potential insights buried within these data.

The proposed EvoMap framework can unlock this potential by revealing the trajectories of firms’ competitive positions over time. These trajectories contribute multiple new insights to market structure and competitive analysis. First, by providing a forward-looking perspective on competition, they can help managers to detect competitive threats and opportunities early. Naturally, maintaining competitive advantages in dynamic environments is challenging (Eisenhardt 1989). Thereby, recognizing competitive threats early (i.e., while they are small) is critical to counter them (Peteraf and Bergen 2003). Doing so, however, is often a challenging task, as firms need to consider not only their closest peers but a broad range of potential competitors (Bergen and Peteraf 2002). Such broad-based competitor identification is especially challenging within a dynamic competitive landscape, as competition can come in many forms and from many directions (Levitt 1960, Peteraf and Bergen 2003). The trajectories estimated by EvoMap can help identify such threats by revealing other firms with converging trajectories. Thereby, EvoMap can jointly identify the trajectories of hundreds of firms (as demonstrated in our empirical application).

Beyond identifying competitors, the revealed trajectories can also help monitor competitors’ actions and alert managers about their rivals’ strategic intentions, such as changes in their positioning strategy. When formulating one’s positioning strategy, a firm needs to consider its competitors’ current positions and also their potential actions, for example, whether they move in a similar direction or not (Ning and Villas-Boas 2021). Monitoring competitors’ trajectories can offer such indications and thus inform marketing strategies. Finally, once a firm has decided to reposition itself, the firm can monitor its trajectory to evaluate the effectiveness of its actions.

The visual representation in maps provides an easy-to-understand common language that facilitates communication across different roles within a firm. For instance, one could integrate dynamic market structure maps into existing analytics dashboards, which are a primary basis for decision making in the industry (Wedel and Kannan 2016). With such dashboards, decision makers across a firm could easily and frequently monitor market developments.

As EvoMap is a flexible framework, one can further apply it to many other empirical settings beyond the one presented in this article. For instance, one could infer firms’ trajectories from changes in their ad copies or changes in the content on their websites. Such data are regularly archived and often made accessible, for instance, by competitive intelligence platforms. Modern data science provides ample means to identify and measure relationships among firms within such (often unstructured) data. For instance, (pretrained) embeddings allow market researchers to measure the similarity among texts, images, speech, or a combination thereof. Paired with such techniques, EvoMap might be used to study, for example, evolving product designs, firm communication, or brand logos.

Beyond firm-generated content, consumers’ digital footprints offer a plethora of suitable data sources to study evolving market structures. Some examples are search logs, social media posts, and customer reviews. Because such data are often available historically and in real time, they create additional opportunities to study evolving relationships among firms, brands, products, and consumers. We also see potential applications of EvoMap at the intersection of marketing and other domains. For instance, political parties communicate their positioning toward society in manifestos or through members’ speeches. Such communication is often archived historically and could reveal party trajectories. Researchers could also use EvoMap in bibliometrics to study the evolution of scientific fields or individual journals’ positions.

Future research could also explore different ways to link map positions (and changes therein) to firm attributes (e.g., how they describe their products). Doing so could help to understand the underlying market’s structure and its evolution even further. Another avenue might be to investigate why firms in some industries exhibit more dynamic positions than others. As our empirical study showed, there exists substantial variation in the length of firms’ trajectories. Identifying the drivers and consequences of these dynamics could further help managers develop an effective (re)positioning strategy.

Naturally, EvoMap is not free of limitations. First, EvoMap is subject to the limitations of the particular static mapping method used. To mitigate this limitation, we designed EvoMap as a flexible framework that can easily accommodate various static mapping methods and implemented it for three very different methods (i.e., metric MDS, Sammon Mapping, and t-SNE). Second, the quality of EvoMap’s dynamic market structure maps is bounded by the quality of the used data. Although EvoMap offers several innovations, such as noise cancellation, it still depends on valid relationship data. And third, EvoMap introduces two hyperparameters to the mapping process ($\text{α}$ and $p$) that market researchers need to consider in their analysis. However, we (1) investigate and document their impact on map quality and (2) offer a tuning scheme to set them appropriately for the particular empirical setting and data at hand.

Methodologically, EvoMap provides the foundation for further research into mapping the dynamics of competitive relationships. Such research could, for instance, pair EvoMap with additional mapping methods, with more sophisticated optimization procedures (such as majorization techniques); or embed the estimated trajectories into models of consumer choice (for instance, to augment dynamic market structure maps with evolving consumer ideal points); or study how the trajectories of firms that compete in multiple submarkets (e.g., Amazon) evolve differently in those submarkets (Ringel 2022). Although recent research in marketing shows a growing interest in modeling and understanding the dynamics in high-frequency data (Du and Kamakura 2012, Schweidel and Moe 2014, Xiong and Bharadwaj 2014, Ma et al. 2015, Puranam et al. 2017, Zhong and Schweidel 2020), the dynamics of relationships among market actors inherent in such data have received less attention. We hope to open a new avenue for further research in this direction.

Appendix A. Data-Generating Process and Hyperparameter Selection for Simulation Study

Ideally, the simulated competitive relationship matrices should represent various market structures with different submarkets and a wide variety of possible inter- and intrasubmarket relationships. To create such variety in our simulation study, we formulate the following data-generating process (DGP):

First, we assign each firm $i\in I$ with uniform probability to one of $k$ submarkets, where $n=\left|I\right|$ and $k\in {\mathrm{\mathbb{N}}}^{+}$. Then, for each submarket $m\in \{1,\dots ,k\}$, we draw the positions in the d-dimensional attribute space from a Gaussian distribution $\mathcal{N}({\mu }_m, {\sigma }_{\mathit{sub}}^2)$ centered at the submarket mean ${\mu }_m$ with standard deviation ${\sigma }_{\mathit{sub}}$. Within the attribute space, we place the submarkets’ means with uniform probability within the edges of a d-dimensional hypercube with unit length. We thus generate pairs of closer and more distant submarkets and obtain the simulated positions ${\mathit{X}}_{\mathit{sim}}\in {\mathrm{ℝ}}^{n\times d}$. Such a process is flexible enough to simulate competitive relationship matrices for a broad class of potential market structures (e.g., small/large markets with few/many strongly/weakly connected submarkets). At the same time, the proposed DGP is parsimonious because it requires only a small number of meaningfully interpretable simulation parameters.

Next, we modify the simulated positions according to one of the following scenarios:

• Scenario I emergence of a new submarket: A fraction ($\rho$) of firms are randomly assigned to a newly formed submarket and, over $T$ periods, change their positions toward the new submarket.

• Scenario II shifts in positioning: A fraction ($\rho$) of firms are randomly assigned to a different yet existing submarket and, over $T$ periods, gradually shift their positions toward their assigned submarket.

• Scenario III market entry: A fraction ($\rho$) of firms enter the market at random positions in the attribute space. Over $T$ periods, each new entrant gradually shifts toward its assigned submarket.

To make our simulation more realistic, we implement two other phenomena that occur in real-world empirical data: nongradual evolution and noise. Our intuition is the following. First, we do not expect changes in firm positions to occur gradually and simultaneously over time. We, therefore, vary the speed at which each firm changes its position during our simulation. Specifically, we draw $T$ (the number of periods over which a firm changes its position) randomly for each firm from a zero-truncated Poisson distribution. As a result, some firms will adjust their positions within each simulation iteration over a few periods (i.e., quickly and less gradually), and others over more periods (i.e., slowly and more gradually). We set the Poisson parameter to eight for all simulation iterations. Second, empirical measurements of competitive relationships can be subject to noise because of the nature of the data source or the collection process. We, therefore, add varying degrees of white noise drawn from $\mathcal{N}(0, {\sigma }_{\mathit{temporal}}^2)$ to each firm’s simulated position at each period before taking distances. We estimate the maps using the noisy relationship matrices but evaluate them against the simulated positions without noise. Table A.1 provides an overview of simulation parameters, their underlying market structure characteristic, the values assigned to the parameters, and the total number of simulation iterations. We chose the parameter space such that it covers a broad spectrum of possible market structures.

Table A.1. Simulation Parameter Space

Table A.1. Simulation Parameter Space

Simulation parameter	Label	Market structure characteristic	Values
n	Number of firms	Market size	50	100	250
d	Number of dimensions	Market complexity	4	8	16
k	Number of submarkets	Market fragmentation	4	8	12
${\sigma }_{\mathit{sub}}$	Within submarket standard deviation	Degree of submarket separation	0.050 (strong)	0.075 (medium)	0.150 (weak)
${\sigma }_{\mathit{temporal}}$	Temporal noise	Smoothness of evolution	0.010 (low)	0.025 (medium)	0.050 (high)
$\rho$	Share of firms affected	Market dynamism	0.050 (low)	0. 100 (medium)	0.150 (high)
Number of combinations	$3^6 = 729$
Number of evolution scenarios	3
Number of simulation iterations	$729 \times 3 = 2,187$

---

Notes. The table lists all simulation parameters. We simulate each scenario for each combination. In total, we conduct 2,187 simulation iterations (729 combinations for each of the three evolution scenarios).

As we outlined in Section 3.3, running EvoMap requires setting two hyperparameters ($\alpha$ and $p$). According to our tuning strategy, we would usually first evaluate several combinations of these two hyperparameters and then manually fine-tune them. Unfortunately, such an elaborate procedure is not feasible for 2,187 simulation iterations. Therefore, we adopt the following approach: In each simulation iteration, we evaluate the following $3\times 5=15$ hyperparameter combinations for EvoMap: $\alpha \in \left\{{10}^{-5}, {10}^{-4}, {10}^{-3}\right\}, p\in \left\{1,\dots ,5\right\}$; and we pick the combination that yields the best result across all metrics. To identify this best combination, we rank the 15 combinations according to our four metrics and then pick the one with the lowest (i.e., best) rank across all four metrics.

Likewise, running dynamic t-SNE requires setting its hyperparameter $\lambda$, which performs a similar task as EvoMap’s hyperparameter $\alpha$ (i.e., both $\alpha$ and $\lambda$ balance the static versus temporal component of the respective cost functions). Thus, we also run dynamic t-SNE for the same three values $\lambda \in \left\{{10}^{-5}, {10}^{-4}, {10}^{-3}\right\}$ and pick the best result for each simulation iteration accordingly. For larger values of $\lambda$, we find that static goodness-of-fit (measured via Hit-Rate) drops substantially for dynamic t-SNE.⁴ We set the remaining parameters of dynamic t-SNE (e.g., its optimization settings) to the default values suggested by its authors.

Figure A.1 shows the hyperparameter distributions for all 2,187 simulation iterations. Notably, for most cases (>98%), the best solution is obtained with $p$ greater than one. In these cases, EvoMap’s additional smoothing penalties lead to better solutions.

Appendix B. Simulation Results for Metric MDS and Sammon Mapping

We also evaluate EvoMap in our simulation study paired with alternative static mapping methods and present the results in Table B.1. We exclude t-SNE-specific alternatives in this comparison (i.e., Dynamic t-SNE, sequential and fixed initialization). As for EvoMap paired with t-SNE, we also run each additional alternative of EvoMap for multiple hyperparameter combinations. Specifically, we run EvoMap (with MDS/Sammon) for $3\times 3=9$ combinations: $\alpha \in \left\{{10}^{-2}, {10}^{-1},5\times {10}^{-1}\right\}$ and $p\in \left\{1,\dots ,3\right\}$. Figure B.1 shows the distribution of the selected hyperparameter values. Note that the range of suitable values for EvoMap’s hyperparameters depends on the chosen mapping method (and its static cost function). The results in Table B.1 show that in line with our previous findings for EvoMap paired with t-SNE, pairing EvoMap with MDS or Sammon Mapping also produces much better solutions than other approaches.

Table B.1. Simulation Results for MDS and Sammon Mapping

Table B.1. Simulation Results for MDS and Sammon Mapping

Dynamic mapping approach	MIS-ALGN	PERS	C-CORR	HIT-RATE
Panel A: MDS
(I) EvoMap (MDS)	0.067	0.894	0.748	0.546
	(0.078)	(0.099)	(0.318)	(0.137)
(II) Ex post aligned MDS	0.505	−0.389	0.253	0.525
	(0.548)	(0.201)	(0.373)	(0.140)
(III) Independent MDS	2.428	−0.480	−0.130	0.525
	(1.360)	(0.102)	(0.201)	(0.140)
Panel B: Sammon mapping
(I) EvoMap (Sammon)	0.094	0.868	0.718	0.598
	(0.155)	(0.137)	(0.317)	(0.135)
(II) Ex post aligned Sammon	5.023	−0.488	0.082	0.586
	(11.320)	(0.114)	(0.298)	(0.140)
(III) Independent Sammon	6.335	−0.488	−0.090	0.586
	(11.647)	(0.089)	(0.209)	(0.140)

---

Notes. Reported metrics are averages over 2,187 simulations as outlined in Table A.1 (729 combinations and three evolution scenarios). Standard deviation in parentheses. Best result per metric marked in bold. MIS-ALGN (misalignment), misalignment of successive maps, measured as the average length of all movement in positions; PERS (persistence), persistence of identified trajectories, measured as the average Pearson correlation coefficient of successive changes in positions; C-CORR (change correlation), average Pearson correlation coefficient of trajectory length on the map versus trajectory length in the simulated positions; HIT-RATE (Hit-Rate), average 10-NN Hit-Rate of nearest neighbor recovery, adjusted for random agreement.

Appendix C. Assessing Overfitting

EvoMap seeks to identify underlying trends in the changes in firms’ competitive positions. Doing so requires separating trends from noise or, stated differently, avoiding overfitting individual periods. Overfitting naturally arises when fitting maps to each period independently, as the resultant maps will adjust to all (even minor) changes in every period, including temporary noise. For EvoMap, increasing the hyperparameter p sets stronger constraints on the resultant trajectories, linking the estimated positions across an increasing number of successive periods. Thereby, the resultant maps should be less sensitive to changes in individual periods, thus, preventing overfitting.

To test this hypothesis, we randomly split the data in each simulation iteration into two parts: estimation and hold-out data. We split along the time axis, such that the full relationship matrix in each period between the first and last period either belongs to the hold-out or the estimation data. We then estimate the sequence of maps using only the relationship matrices in the estimation data. To test for overfitting, we predict the positions on the map for the hold-out data using the estimated trajectories. Specifically, to derive firm i’s position in any period in the hold-out data, we use linear interpolation between the nearest preceding and succeeding map position in the estimation data. Doing so predicts the map position along the estimated trajectories and tests the extent to which these trajectories capture the underlying evolution trends. For evaluation, we compute the average (adjusted) Hit-Rates for the predicted and estimated maps, respectively (see Table C.1). As expected, independent mapping (even when aligned ex post via Procrustes analysis) overfits most heavily. EvoMap, in contrast, exhibits no sign of overfitting: The average difference between the Hit-Rate on the estimation and hold-out data are below one percentage point.

Table C.1. Hold-Out Validation Results

Table C.1. Hold-Out Validation Results

Dynamic mapping approach	HIT-RATE (estimation data)	HIT-RATE (hold-out data)	HIT-RATE % change
(1) EvoMap (t-SNE)	66.093	65.495	−0.595
	(10.588)	(10.658)	(1.955)
(2) Dynamic t-SNE (Rauber et al. 2016)	44.109	42.809	−1.296
	(13.944)	(14.083)	(2.822)
(3) Ex post alignment (t-SNE)	63.780	60.447	−3.330
	(12.039)	(13.416)	(3.179)
(4) Sequential initialization (t-SNE)	63.106	60.900	−2.202
	(12.301)	(13.414)	(2.719)
(5) Fixed initialization (t-SNE)	63.795	60.366	−3.426
	(12.032)	(13.513)	(2.970)
(6) Independent mapping (t-SNE)	63.780	58.443	−5.333
	(12.039)	(13.146)	(3.354)

---

Notes. Average adjusted 10-NN Hit-Rate values in percentage points across 2,187 simulation iterations. Standard deviation in parentheses.