AI-Driven Human Task Substitution: Evidence from Google Translate and ChatGPT

1. Introduction

Artificial intelligence (AI) is increasingly capable of performing nonroutine cognitive tasks traditionally carried out by skilled knowledge workers, raising the prospect of substantial disruption across a wide range of industries (e.g., Agrawal et al. 2018, 2023; Felten et al. 2021, 2023; Fedyk et al. 2022; Brynjolfsson et al. 2025). Following the release of ChatGPT, public and academic discourse around the substitution potential of AI has intensified. Recent studies suggest that AI could plausibly substitute for a large share of knowledge work tasks, with estimates ranging from 35% to 80%, though these projections vary widely in scope and methodology (Fedyk et al. 2022, Agrawal et al. 2023, Brynjolfsson et al. 2025). Despite this growing body of work, there is limited retrospective empirical evidence on how AI’s impact varies across different types of knowledge worker tasks. While a small number of recent studies have begun to explore heterogeneity at the level of job clusters (e.g., Hui et al. 2024, Demirci et al. 2025), task-level evidence on differential substitution effects remains scarce (Fedyk et al. 2022, Agrawal et al. 2023, Brynjolfsson et al. 2025, Doshi and Hauser 2024, Jia et al. 2024).

Earlier waves of automation primarily substituted for human labor in occupations dominated by routine tasks, such as accounting, data entry, and assembly line work (Autor et al. 2003, Acemoglu and Restrepo 2020). By contrast, the nonroutine cognitive tasks that characterize much of contemporary knowledge work are more complex, often requiring judgment, interpretation, and contextual understanding (Brynjolfsson et al. 2018, Frank et al. 2019, Tong et al. 2021, Agrawal et al. 2023). Several factors help explain why retrospective evidence on AI’s substitution effects in these domains remains limited. First, fine-grained data on the adoption and use of AI technologies is typically unavailable (Raj and Seamans 2018, Frank et al. 2019, Acemoglu et al. 2022). Second, within-firm task structures often require complementary investments in technology and human capital, complicating efforts to isolate the effects of AI itself (Choudhury et al. 2020, Brynjolfsson et al. 2021). Third, the gradual and uneven diffusion of AI across firms and industries makes it difficult to cleanly identify causal impacts on workers and organizations (Brynjolfsson et al. 2021).

In this study, we address these limitations by responding to calls for more fine-grained empirical investigations into the effects of AI on human tasks (Frank et al. 2019) and for clearer distinctions between AI technologies with different capabilities (Acemoglu et al. 2022). We provide empirical evidence on how two sequential forms of AI—a predictive, task-specific system (PAI) and a more recent, broad, general-purpose generative system (GAI)—affected the demand for human knowledge work. Our analysis focuses on whether and how these effects differ between analytical tasks and tasks requiring intuitive and creative judgment.

Our study complements recent work by Frey and Llanos-Paredes (2025), who document the long-run effects of Google Translate on translator employment and the demand for foreign language skills at the aggregate labor market level. Although their findings establish that predictive AI reduced overall demand for translation-related labor, they leave open the question of which specific tasks drove this substitution, and whether conclusions drawn from the predictive AI era generalize to newer forms of AI. We address these gaps in three ways. First, by comparing predictive and generative AI shocks within the same empirical context, we identify how the magnitude and boundaries of AI substitution have shifted over time, rather than simply documenting differences in performance across tools. Second, we disaggregate effects at the task level, distinguishing between analytical (regular translation) and intuitive (transcreation) tasks, revealing heterogeneity that occupation-level analyses necessarily obscure. Third, by using transaction-level data from an online labor market (OLM), we provide granular evidence on the immediate substitution effects of successive AI technologies.

The empirical evidence we offer is anchored in two sequential natural experiments in the context of language translation, a domain populated by skilled knowledge workers whose tasks involve the generation, processing, and transformation of information. The first natural experiment is the deployment of Google Neural Machine Translation (GNMT) as a backend upgrade to the existing Google Translate product for specific language pairs beginning in November 2016. Although Google Translate had been publicly available and widely used for years, GNMT replaced its earlier phrase-based engine with a neural architecture, constituting a discrete and substantial improvement in translation quality. GNMT is a sophisticated neural machine translation system and exemplifies a predictive AI designed for a narrowly defined, sequence-to-sequence task. Its staggered rollout constituted an exogenous shock that suddenly and unexpectedly made high-quality machine translation widely accessible at zero marginal cost.

The second natural experiment is the release of ChatGPT by OpenAI in November 2022. ChatGPT represents a qualitatively different form of AI: a broad, general-purpose generative system capable of performing translation alongside a wide range of other cognitive tasks, such as ideation, synthesis, and analysis (Korinek 2023). Taken together, these two sequential shocks allow us to examine not only whether AI substitutes for human translation work, but also how the scope of substitution changes as AI architectures evolve. This design provides a retrospective perspective that complements forward-looking assessments of AI’s labor-market implications (e.g., Raj and Seamans 2018, Frank et al. 2019) and enables a task-level analysis of substitution that is difficult to obtain in studies focused on a single technology or period (e.g., Brynjolfsson et al. 2025).

With the release of ChatGPT, questions about whether generative AI systems differ meaningfully from earlier predictive AI systems in their labor market effects became more salient in public discourse. In particular, observers began to ask whether advances in AI architecture expand the set of tasks for which human labor is substitutable, especially among nonroutine cognitive tasks that require interpretation, judgment, or creative adaptation. While both predictive and generative AI systems can perform analytical tasks, generative AI’s training and architecture may allow it to perform tasks that rely more heavily on contextual understanding and meaning. Despite widespread discussion of these possibilities, retrospective empirical evidence comparing how different generations of AI affect distinct task types remains limited.

To structure our investigation, we ask three related questions. First, how do AI technologies affect overall labor demand for knowledge work? Second, does the magnitude of this effect vary across task types, specifically between analytical tasks and tasks requiring intuitive or creative judgment? Third, do different AI architectures (predictive versus generative) produce distinct substitution patterns across these tasks? Together, these questions allow us to assess not only whether AI substitutes for human labor but whether conclusions drawn from earlier AI waves generalize to newer forms of AI. A central contribution of our analysis is not only documenting whether substitution occurs but quantifying how substitution magnitudes differ across task types and AI generations.

To answer these questions, we examine the effects of GNMT and ChatGPT on the demand for human language translation in a major OLM, which provides a standardized and complete archive of transactions (Agrawal et al. 2016, Horton and Tambe 2025). We distinguish between two types of translation tasks: (i) regular translation, which involves converting text from one language to another, and (ii) transcreation, which requires creative and cultural adaptation to preserve meaning, tone, and style for a new audience. Transcreation is common in settings such as marketing, poetry, and literary translation and entails stylistic reinvention and cultural sensitivity (Holland and Quinn 1987, Glăveanu et al. 2013).

Employing a difference-in-differences design, we find that GNMT reduced demand for regular translation services by approximately 9.1% within seven months of launch, having no statistically significant effect on transcreation tasks. ChatGPT, by contrast, exhibits a markedly different pattern: demand declines substantially for both regular translations (21.1%) and transcreations (29.6%), indicating comparable substitution effects across analytical and intuitive tasks. Supplemental analyses reveal an additional distinction across AI generations. Whereas GNMT’s impact does not extend to tasks requiring deep domain-specific knowledge, ChatGPT is associated with significant reductions in demand for specialized technical translations as well.

Taken together, these findings suggest that, whereas both predictive and generative AI substitute for nonroutine cognitive work, they do so in qualitatively different ways. In the predictive AI era, substitution was concentrated in analytical tasks, leaving intuitive and creative tasks relatively insulated. In the generative AI era, this boundary no longer holds. As a result, conclusions about which types of knowledge work are resilient to automation (drawn from earlier AI waves) do not straightforwardly generalize to newer forms of AI.

These findings are particularly relevant given the accelerating pace of AI development and our task-level focus within an occupation. Occupations, firms, and industries can be understood as bundles of tasks: that is, distinct units of work that vary in their susceptibility to automation (Autor et al. 2003, Acemoglu and Autor 2011, Garicano and Wu 2012). AI technologies are, therefore, unlikely to substitute for entire occupations at once, instead affecting specific tasks depending on the alignment between a technology’s capabilities and task demands. As a result, even within the same occupation or firm, workers may face very different substitution risks depending on the bundle of tasks they perform.

Our results show that this task-level heterogeneity is shaped not only by whether tasks are nonroutine but also by whether they are primarily analytical or intuitive, and by the generation of AI technology being deployed. In particular, task allocations that appeared relatively insulated from automation in the predictive AI era become exposed under generative AI. This task-level lens is, therefore, essential for understanding how advances in AI reshape the boundaries of substitution over time, expanding the range of tasks that can be automated, and limiting the extent to which inferences drawn from earlier automation waves generalize to newer forms of AI.

2. Background Literature

2.1. The Substitution Effect of AI on Human Labor

Developments in AI have the potential to substantially affect firms, industries, and labor markets. Within management research, scholarly discourse has focused on how quickly and to what extent AI technologies may reshape competition, employment structures, and firms’ human capital (Choudhury et al. 2020, Tschang and Almirall 2021, Krakowski et al. 2023). Much of the early work on AI’s labor market implications has been forward looking, relying on theoretical models or expert assessments to forecast the potential scope of AI-driven substitution (e.g., Pajarinen and Rouvinen 2014, Brzeski and Burk 2015, Frey and Osborne 2017, Webb 2019, Felten et al. 2023, Eloundou et al. 2024).

More recent studies have begun to provide retrospective evidence by examining hiring outcomes. For example, Acemoglu et al. (2022) study firm-level online vacancy data and show that firms adopting AI reduce hiring for certain non-AI positions, altering skill requirements in remaining postings. Similarly, Fedyk et al. (2022) analyze resume data from large audit firms and find that increased AI investment is associated with a decline in accounting employment over time. Although these studies offer important insights into how AI adoption correlates with changes in labor demand, they focus primarily on occupation-level hiring outcomes and rely on observational variation that may be influenced by multiple confounding factors.

In a related study, Frey and Llanos-Paredes (2025) document a decline in aggregate demand for translators following the diffusion of Google Translate, providing compelling long-run evidence of labor displacement associated with predictive AI, estimating that Google Translate adoption reduced translator employment growth by 0.84 percentage points (instrumental variable estimate) and resulted in approximately 28,000 fewer positions over the 2010–2023 period. However, their occupation-level approach leaves open the question of which specific tasks within translation drove this substitution and whether similar patterns extend to newer generations of AI. Our approach complements this work by focusing on tasks within an occupation and by leveraging two natural experiments to isolate substitution effects more directly, thereby offering a more granular perspective on how AI technologies reshape labor demand at the task level (Frank et al. 2019, Acemoglu et al. 2022).

Distinguishing between different classes of AI systems is particularly important for understanding these dynamics. Although both PAI and GAI fall under the broader umbrella of AI, they differ in architectural scope and the types of tasks for which they are optimized. PAI systems are typically designed for convergent, sequence-to-sequence problems. The GNMT system we study is a highly sophisticated example of such an approach, optimized to predict accurate translations within a well-defined output space. Although neural machine translation models are technically generative in that they produce new text, their architecture is geared toward convergent objectives, and this has historically limited their effectiveness in tasks requiring divergent thinking, creative judgment, or deep contextual interpretation (Argote et al. 2021, He et al. 2024).

Generative AI systems, by contrast, are built on large language models trained on diverse corpora and designed to generate a wide range of outputs across contexts. Tools such as ChatGPT or DALL-E can produce text or images that vary in tone, style, and form, and this has raised questions about whether such systems may affect a broader set of nonroutine cognitive tasks. Whereas these capabilities suggest potential differences in labor market impact relative to earlier AI systems, empirical evidence comparing how distinct AI architectures translate into differential substitution effects remains limited.

Understanding AI-driven substitution, therefore, requires moving beyond occupation-level analyses to examine the tasks that constitute human work. Occupations comprise heterogeneous bundles of tasks that vary in their susceptibility to automation, and task characteristics play a central role in shaping the substitution risk. By focusing on tasks rather than occupations, we follow established frameworks that emphasize how technological change reallocates work within jobs rather than eliminating jobs wholesale. In the next section, we build on this task-based perspective to categorize knowledge worker tasks and to examine how predictive and generative AI systems differentially affect them.

2.2. Knowledge Worker Tasks and the Implications of AI

Knowledge workers typically perform a range of tasks that differ in their susceptibility to automation and substitution. Prior research commonly distinguishes tasks along two dimensions: (i) routine versus nonroutine, capturing whether tasks involve explicitly repetitive procedures or require adaptation and judgment, and (ii) manual versus cognitive, reflecting whether tasks rely primarily on physical or intellectual work (Autor et al. 2003, Acemoglu and Autor 2011).

Routine tasks, both manual and cognitive, follow explicit rules that can be readily specified in code and have long been subject to automation through pre-AI technologies (Autor 2014). Routine manual tasks such as product assembly, painting, and welding have been widely automated using industrial machinery (Acemoglu and Restrepo 2020, Dixon et al. 2021). Routine cognitive tasks, including billing, accounting, and data entry, have similarly been automated through software systems that execute preprogrammed instructions (Autor et al. 2003). Nonroutine tasks, by contrast, involve greater complexity and uncertainty and often require problem solving, judgment, and contextual interpretation. Examples of nonroutine manual tasks include driving in traffic or preparing a meal, whereas nonroutine cognitive tasks include medical diagnosis, legal reasoning, marketing campaign creation, scientific research, and translation. Until recently, such nonroutine tasks were widely viewed as relatively insulated from automation (Brynjolfsson et al. 2018).

Recent scholarship suggests that AI technologies may increasingly substitute for some nonroutine tasks (Brynjolfsson et al. 2018, Frank et al. 2019, Acemoglu et al. 2022, Fedyk et al. 2022), including medical diagnosis (Wang et al. 2019), wind turbine maintenance (Franko et al. 2020), and auditing (Fedyk et al. 2022). However, the substitution potential of AI is argued to be more limited when nonroutine cognitive tasks rely heavily on emotional intelligence, cultural interpretation, or other forms of context-dependent judgment (Faraj et al. 2018). This limitation reflects the fact that humans process not only codifiable inputs but also intangible and context-dependent cues such as emotions, cultural norms, and social subtleties that are difficult to formalize and encode (Marcus and Davis 2019, Wu et al. 2020). As a result, some nonroutine cognitive tasks have been viewed as more resistant to automation at least in the near term (Agrawal et al. 2018, Autor and Salomons 2018, Acemoglu et al. 2022).

Beyond the analytical–intuitive distinction, a task’s susceptibility to automation may also depend on its reliance on deep, domain-specific knowledge. Fields such as law, medicine, and engineering operate with specialized vocabularies and high-stakes accuracy requirements, which early AI systems that were often trained on broad, general-purpose corpora were thought to handle imperfectly. This has given rise to a perceived knowledge barrier to substitution. Whether the transition from narrow, task-specific AI to broad, general-purpose generative AI has altered the permeability of this barrier remains an open empirical question.

A growing body of research examines the performance of generative AI systems in tasks involving creativity, interpretation, and judgment. Laboratory and field experiments show that generative AI can enhance output quality in tasks such as business ideation and consulting (Girotra et al. 2023, Boussioux et al. 2024, Doshi and Hauser 2024, Dell’Acqua et al. 2026), whereas quasi-experimental evidence from creative industries suggests effects on professional behavior and strategy (Peukert et al. 2024). Studies in professional services, including law, document substantial productivity gains associated with AI use (Schwarcz et al. 2026). At the same time, the extent to which these capabilities translate into substitution in real-world labor markets remains underexamined. Indeed, recent work highlights that large language models underlying generative AI systems are better at processing literal text than interpreting broader social or cultural context, and this may limit their ability to interpret meaning in culturally specific settings (NLLB Team 2024). For example, linguistic conventions such as the use of politeness to convey irony may be difficult for such systems to capture reliably.

Taken together, these perspectives suggest that different classes of AI may exhibit distinct substitution patterns across tasks with important implications for occupations, firms, and industries. Because occupations are composed of heterogeneous bundles of tasks (Autor et al. 2003, Acemoglu and Autor 2011, Garicano and Wu 2012), workers performing different task mixes may face very different automation risks even within the same job category.

In this paper, we examine how predictive and generative AI differentially affect nonroutine cognitive tasks. We disaggregate such tasks into those that are primarily analytical and those that rely more heavily on intuitive or creative judgment (Autor et al. 2003, Spitz-Oener 2006). Whereas intuitive tasks also involve substantial cognitive effort, they differ from analytical tasks in their objectives and structure. Analytical tasks are typically convergent and rule-based, requiring the application of logic and learned patterns to arrive at an optimal or probabilistically correct solution. Intuitive tasks, by contrast, are often divergent and goal-oriented, requiring the generation of novel outputs that achieve emotional, cultural, or stylistic objectives in a new context.

This distinction maps onto the architectural differences between the AI systems we study. Predictive AI systems such as GNMT are optimized for convergent, sequence-to-sequence problems and perform well within well-defined output spaces. Generative AI systems such as ChatGPT are designed to produce a broader range of outputs and may, therefore, affect a wider set of tasks. Whether and how these architectural differences translate into differential substitution effects is ultimately an empirical question, one that we examine in the context of translation, a domain involving nonroutine cognitive work and previously assessed as facing moderate automation risk (Frey and Osborne 2017, Felten et al. 2021).

2.3. Research Questions

In summary, we address three interlinked research questions designed to clarify how the scope of AI-driven task substitution has evolved. First, what is the substitution effect of AI on nonroutine cognitive tasks? We address this by examining whether and to what extent AI technologies substitute for human labor within the domain of language translation. Second, does this substitution effect differ across generations of AI technologies? We examine this by comparing the impacts of predictive AI (GNMT) and generative AI (ChatGPT) within the same empirical setting. Third, does the effect vary systematically across task types, specifically between nonroutine cognitive tasks that are primarily analytical and those that rely more heavily on intuitive or creative judgment? We examine this by distinguishing between regular translations and transcreations.

Taken together, retrospective empirical evidence on these questions allows us to assess not only whether AI substitutes for human knowledge work but also whether conclusions drawn from earlier AI waves generalize to newer forms of AI. Addressing these questions jointly provides a task-level perspective on how advances in AI architecture reshape the boundaries of substitution for knowledge workers.

3. Empirical Contexts

3.1. Comparing GNMT and ChatGPT

We compare GNMT, as an exemplar of a highly specialized PAI system with ChatGPT, as an exemplar of a broad, general-purpose GAI system. GNMT is designed to perform a narrowly defined, convergent task: producing accurate translations by predicting the most likely output sequence for a given input. ChatGPT, by contrast, is based on a large language model trained to generate text across a wide range of contexts and prompts. As a result, it can perform translation alongside other language-related tasks, such as summarization, synthesis, and text generation.

These architectural differences prompt speculation that generative AI systems may affect a broader set of tasks than earlier predictive systems, particularly tasks involving interpretation, tone, or contextual adaptation. At the same time, such differences do not mechanically imply broader labor substitution as demonstrated performance does not necessarily translate into displacement in market settings. Whether and how these distinct architectures lead to different substitution patterns, therefore, remains an empirical question.¹

Following the release of ChatGPT, attention to the labor market implications of AI intensified relative to earlier periods dominated by predictive AI technologies (Kochhar 2023, World Economic Forum 2023). Popular and practitioner-oriented accounts often emphasize the potentially wide-ranging impact of generative AI on work (e.g., Accenture 2023). Our study complements this discourse by providing retrospective, task-level evidence that allows us to assess how differences in AI architecture translate into observable substitution effects in a real labor market.

3.2. GNMT and ChatGPT as Shocks to Translation Tasks

Google Translate has been publicly available since 2006 and achieved widespread consumer adoption well before 2016, serving as one of the most widely used translation tools globally (Yamada 2019). However, early versions of Google Translate (as did other machine translation technologies of that era) relied on rule- and phrase-based statistical approaches that produced outputs substantially inferior to human translation (Wu et al. 2016). The emergence of neural machine translation (NMT), which incorporates broader grammatical and semantic context, led to marked improvements in translation quality and fluency (Luong and Manning 2015, Wu et al. 2016, Jia et al. 2019, Yamada 2019). Prior to Google’s upgrade, however, access to high-quality NMT systems was limited. In late 2016, Google deployed GNMT as a backend architectural upgrade to its existing Google Translate product, replacing the earlier phrase-based system with a neural machine translation engine.² Our treatment, therefore, corresponds to the deployment of GNMT within Google Translate, a discrete shift from phrase-based to neural machine translation rather than to the introduction of Google Translate per se. This upgrade substantially improved translation quality and represented a sudden and unexpected increase in the availability of accurate, low-cost machine translation (Wu et al. 2016, Jia et al. 2019).³ Given Google Translate’s already widespread adoption at the time of the GNMT upgrade, the sudden improvement in translation quality raised immediate concerns about the potential impact of machine translation on professional translators (Lewis-Kraus 2016, Yoon 2017, Massey and Kiraly 2019).

Although task-specific AI systems such as GNMT perform well in regular translation contexts such as news articles, essays, and technical documents, their applicability is argued to be more limited for tasks requiring adaptation to cultural, stylistic, or emotional nuance (Pedersen 2014, Vieira 2020). Linguists use the term “transcreation” to describe translation tasks that require intuitive and creative reinterpretation rather than literal conversion, such as advertising copy or poetry, for which understanding cultural and emotional context is central (Pedersen 2014).

The release of ChatGPT in late 2022 constituted a second, qualitatively different shock. ChatGPT is a general-purpose conversational AI system built on a large language model and capable of performing translation alongside a wide range of other language-based tasks. This broader applicability is characteristic of technologies that exhibit general-purpose properties with implications for diffusion and labor market impact (Goldfarb et al. 2023). Its rapid diffusion brought generative AI capabilities to a broad user base in a short period of time. At the same time, contemporaneous assessments raised questions about its reliability, accuracy, and ability to consistently handle culturally specific or high-stakes content (Glorioso 2023, Hunter 2023), concerns that subsequent systematic evaluations have substantiated (Dahl et al. 2024, Yao et al. 2024). As with GNMT, the introduction of ChatGPT, therefore, altered the technological environment faced by translators in ways that were salient, rapid, and largely outside the control of individual workers.

4. Data and Methodology

4.1. Data Sources

To examine the impact of GNMT, we use data from one of the largest OLMs. OLMs have become an increasingly important venue for sourcing human labor in activities such as translation, proofreading, and software development (Chen and Horton 2016). Importantly for our purposes, OLMs provide a complete record of posted tasks and completed transactions, which makes them well suited for assessing how technological shocks affect labor demand (Agrawal et al. 2016, Chen and Horton 2016).

We analyze two distinct time periods corresponding to our natural experiments. To study the effects of GNMT, we focus on January 2016 through May 2017, ending the sample prior to GNMT’s expansion to most language pairs offered by Google Translate in June 2017. This data set comprises 38,573 transactions across treated and control language pairs for which the language pair can be identified. To study the effects of ChatGPT, we analyze the period from January 2022 through May 2023, chosen to provide a comparable window length. This data set comprises 82,277 transactions with identifiable language pairs. Aggregating to the appropriate panel unit—language pair × month for GNMT and language pair × task type × month for ChatGPT—yields 2,482 observations for the GNMT analysis and 4,352 for the ChatGPT transaction regressions (4,301 for ChatGPT revenue regressions, for which a small number of cells drop because no price is observed).

As in most professions, translation involves heterogeneous tasks. To distinguish between different types of translation work, we classify tasks into analytical and intuitive categories corresponding to two commonly recognized forms of translation.⁴ The first category consists of regular translations, which are primarily analytical in nature and involve the literal conversion of text from one language to another. The second category consists of transcreation tasks, which require adaptive, creative, and culturally sensitive judgment, such as the translation of advertising content or poetry (Massey and Wieder 2019). We classify transcreation as intuitive because task success is evaluated based on achieving a desired cultural, emotional, or stylistic effect rather than on literal fidelity. Each job posting in the translation sample is assigned to exactly one task category with no residual or ambiguous classification (see Online Appendix A2 for the full classification protocol).

In supplemental analyses, we further refine this categorization by distinguishing between regular translations that rely primarily on general knowledge and those that require specialized technical knowledge, such as legal, medical, or engineering content.⁵ This yields three task categories: (i) regular translation tasks involving general knowledge, (ii) regular translation tasks involving technical knowledge, and (iii) transcreation tasks. We formally test for differences in substitution effects across these task categories as reported in Section 5.3.

4.2. Identification Strategy

The introduction of GNMT occurred at different points in time across language pairs and constituted a sudden increase in the quality of machine translation.⁶ In November 2016, GNMT became available for eight language pairs, followed by the addition of 33 further language pairs between March and April 2017. Our identification strategy exploits this staggered rollout. The key identifying assumption is that the timing of GNMT’s introduction for specific language pairs was not anticipated by market participants. This assumption is plausible given that neither translators nor clients could foresee the precise sequence or timing of language pair coverage. Accordingly, we treat the GNMT rollout as a quasi-experimental shock that generates plausibly exogenous variation in technology availability across language pairs rather than as a randomized experiment. Because our design is quasi-experimental rather than randomized, identification rests on parallel trends and fixed effects assumptions rather than covariate balance. We assess validity through event study analyses and placebo tests rather than balance tables (see Section 5.1 and Online Appendix A12). To isolate the effect of GNMT, we restrict the sample to January 2016 through May 2017, ending the posttreatment window before GNMT expanded to most language pairs in June 2017.

We conduct the GNMT analysis at the market level, examining how the volume of translation transactions evolves following the introduction of GNMT. Because a substantial share of observations take the value zero (47.3% in the GNMT sample), we estimate fixed effects Poisson pseudo-maximum-likelihood (PPML) models. PPML accommodates zero outcomes naturally and avoids the need for ad hoc transformations required by log-linear ordinary least squares (OLS) specifications (Chen and Roth 2024). Prior work shows that PPML yields consistent estimates under a wide range of heteroskedasticity and data-generating processes, whereas log-linear OLS and related alternatives can be biased or inefficient when variance is misspecified or zeros are prevalent (Gourieroux et al. 1984, Santos Silva and Tenreyro 2006, Cameron and Trivedi 2013, Powell and Seabury 2018). Moreover, unlike fixed effects negative binomial models, PPML permits the inclusion of high-dimensional fixed effects without introducing incidental parameter bias (Allison and Waterman 2002, Santos Silva and Tenreyro 2011, Martin and Pham 2020).

Our baseline specification uses a language pair panel and is given by

where $Y_{it}$ denotes the total number of transactions for language pair $i$ in month $t$. The variable ${\text{PostTreatment}}_{it}$ equals one for observations corresponding to language pairs after the introduction of GNMT. Language pair fixed effects (${\gamma }_i$) control for time-invariant characteristics such as linguistic complexity or baseline demand, whereas time fixed effects (${\delta }_t$) capture market-wide trends common to all language pairs. Standard errors are clustered at the language pair level to account for serial correlation. Following Angrist and Pischke (2009), we do not include control variables that may themselves be affected by the treatment (e.g., prices) as doing so could bias the estimated treatment effect. Event study analyses confirm that treated and control language pairs exhibit parallel pretreatment trends with divergence emerging only after GNMT’s introduction (see Section 5.1 and Online Appendix A12).⁷

The ChatGPT analysis requires a different empirical strategy because ChatGPT affected all language pairs simultaneously. We, therefore, define treatment and control groups based on task composition rather than timing. The treatment group consists of pure play translation jobs—projects focused exclusively on translation—whereas the control group comprises bundled translation tasks in which translation is only one component of a broader assignment (e.g., translation and voiceover, translation and captioning, or research and translation). This comparison allows us to examine whether tasks more directly exposed to generative AI experienced differential impacts relative to tasks for which translation is embedded within a multiskill workflow.

This control group provides a meaningful counterfactual for two reasons. First, whereas generative AI can substitute for the translation component of a task, it cannot fully perform the additional skills required in bundled projects, at least initially, reducing substitution pressure on the overall task. Second, because bundled tasks still include translation, they are subject to similar market-wide trends, allowing us to retain language pairs as the core unit of analysis.⁸ Nevertheless, we recognize that some bundled tasks (e.g., research and translation) include components that may themselves be affected by generative AI. If so, the control group would be partially treated, biasing estimates toward zero. We, therefore, interpret our estimates as conservative.

To further address this concern, we also implement a within-unit panel difference-in-differences design. This approach compares changes in outcomes for a given language pair around the introduction of ChatGPT (e.g., 2022–2023) with changes for the same language pair during a pretreatment placebo period (e.g., 2018–2019). This self-controlled design absorbs all time-invariant heterogeneity and relies on the more plausible assumption of stable year-over-year trends in the absence of treatment.

As in the GNMT analysis, we estimate ChatGPT’s effects using fixed effects PPML models. Because treatment status varies by task type as well as by language pair, the unit of observation is language pair × task type × month. We include language pair × task type fixed effects (${\gamma }_{ij}$), which control for time-invariant differences in demand across language–task combinations, and month fixed effects (${\delta }_t$), which capture market-wide shocks and seasonality common to all observations.

5. Empirical Results

5.1. Baseline Results: AI Technologies’ Impact on Translation Demand

We report summary statistics in the Online Appendix (Online Tables A3-1 through A3-11). Figure 1(a) plots the number of translation transactions before and after the introduction of GNMT, comparing the first cohort of treated language pairs with control language pairs. Prior to GNMT’s introduction, treated and control language pairs exhibit similar demand patterns. Following GNMT’s rollout in November 2016, the two groups begin to diverge: transaction volumes for control language pairs grow more rapidly than those for treated language pairs in the posttreatment period. This divergence, in which the control group grows faster than the treated group (which is growing but at a lower rate), is not a threat to identification but rather the variation that identifies the treatment effect. Because GNMT improved translation quality only for treated language pairs, it is the relative slowdown in treated pair growth, not the absolute level of control group activity, that captures the effect of the technology. During our study period, the online labor market was expanding overall, and our difference-in-differences design is specifically intended to net out this common platform-level growth by comparing relative trajectories between treated and control language pairs.⁹

Figure 1(b) presents analogous evidence for the ChatGPT analysis, comparing transaction patterns between pure play translation tasks and bundled translation tasks. The divergence begins in December 2022, shortly after ChatGPT’s release in November 2022, and becomes more pronounced by May 2023, following the release of GPT-4 in mid-March 2023.

Table 1 reports our main estimates of the effects of GNMT and ChatGPT on translation demand. Both technologies are associated with reductions in translation activity though the magnitude and precision of the estimates differ. GNMT reduces transaction volume by 8.2% $(e^{-0.0853}-1)\times 100$, significant at the 5% level. ChatGPT exhibits a larger effect with a 22.2% reduction in transaction volume ($\beta =-0.251$, p < 0.001).

Table 1. Baseline Estimates: GNMT vs. ChatGPT

For revenue, GNMT yields a negative but imprecisely estimated coefficient, corresponding to a 20.6% reduction ($\beta =-0.231$, p = 0.120). We interpret this estimate cautiously. Revenue in online labor markets is inherently more volatile than transaction counts as individual high-value projects can generate substantial month-to-month variation without corresponding changes in transaction volume. By contrast, ChatGPT’s revenue effect is both economically and statistically significant, corresponding to a 26.1% reduction ($\beta =-0.302$, p < 0.05).

Figures 2 and 3 present event study estimates for GNMT and ChatGPT, respectively. For transaction volumes, pretreatment coefficients remain close to zero with no systematic trend, supporting the parallel trends assumption. The revenue specifications display greater pretreatment variability, particularly in the GNMT case, reflecting the higher inherent volatility of revenue measures relative to transaction counts. To mitigate this noise, we also implement coarsened exact matching (CEM) between treated and control observations as described below and obtain consistent estimates.

In both settings, treatment effects emerge shortly after technology introduction and persist throughout the observation window. Whereas the magnitude of the effects varies over time, the estimates generally increase toward the end of the sample period.

5.2. Heterogeneous Effects by Translation Type

Table 2 reports estimates of the heterogeneous effects of AI technologies across analytical and intuitive translation tasks. To facilitate a direct comparison between GNMT and ChatGPT, the analysis is restricted to the subset of language pairs that are active in both observation periods. We distinguish between regular translations, which involve primarily analytical language conversion, and transcreations, which require creative judgment and cultural adaptation.

Table 2. Heterogeneous Effect of GNMT and ChatGPT on Different Translation Tasks: Regular Translations vs. Transcreations

For GNMT, the estimates indicate a reduction in regular translation activity with transaction volume declining by 9.1% ($\beta =-0.0958$, p < 0.05) and revenue declining by 25.6% ($\beta =-0.296$, p < 0.10). By contrast, ChatGPT is associated with larger and more precisely estimated effects on regular translations, reducing transaction volume by 21.1% ($\beta =-0.237$, p < 0.001) and revenue by 26.2% ($\beta =-0.304$, p < 0.05).

The patterns differ more sharply for transcreation tasks. For GNMT, we find no statistically significant effect on either transcreation transactions ($\beta =0.152$, p = 0.155) or revenue ($\beta =0.290$, p = 0.526). Whereas the point estimates are positive, they are not statistically distinguishable from zero. By contrast, ChatGPT is associated with a substantial reduction in transcreation transactions of 29.6% ($\beta =-0.351$, p < 0.001). The corresponding revenue effect is negative but not statistically significant at conventional levels ($\beta =-0.247$, p = 0.169). This pattern is consistent with a decline in the number of transcreation projects alongside greater variability in project size or scope among remaining tasks.

To assess whether the effects differ statistically across task types, we conduct coefficient comparison tests. For GNMT, the difference between the effects on regular translation and transcreation transactions is statistically significant (${\chi }^2=4.44$, p < 0.05), indicating heterogeneous effects across task types. For ChatGPT, although both task categories experience negative effects, the difference between them is not statistically significant (${\chi }^2=2.42$, p = 0.12), suggesting more similar impacts across analytical and intuitive tasks.

Figures 4 and 5 present event study estimates that illustrate the dynamic evolution of these effects. For GNMT, the reduction in regular translation activity emerges gradually over time, whereas transcreation exhibits no discernible posttreatment shift. For ChatGPT, the decline in regular translation activity occurs immediately following introduction. Transcreation tasks display a change from a nonnegative pretreatment trend to a predominantly negative posttreatment trajectory, albeit with greater month-to-month variability.

Taken together, these results indicate that GNMT’s substitution effects are concentrated in analytical translation tasks, whereas ChatGPT is associated with reductions across both analytical and intuitive tasks with no statistically significant difference in magnitude between them.

5.3. Distinguishing Between General and Technical Translation

We further distinguish between general translations and technical translations, defined as translations of medical, legal, and engineering texts, within the regular translation sample. Table 3 reports the corresponding estimates.

Table 3. Heterogeneous Effect of GNMT and ChatGPT on Different Translation Tasks: General vs. Technical Translation

For GNMT, we find no statistically significant effects on technical translation activity. Transaction volume shows no detectable change ($\beta =-0.0418$, p = 0.681), and the revenue estimate is likewise small and imprecise ($\beta =-0.144$, p = 0.689). By contrast, ChatGPT is associated with substantial reductions in technical translation demand with transactions declining by 25.9% ($\beta =-0.299$, p < 0.001) and revenue declining by 30.7% ($\beta =-0.366$, p = 0.02).

As in the previous section, we formally test whether technical and general translations respond differently to each technology. For GNMT, the difference in effects between technical and general translations is not statistically significant (${\chi }^2=0.19$, p = 0.662), indicating similarly weak effects across these analytical task subcategories. For ChatGPT, both general and technical translations are significantly affected individually, but the difference between them is not statistically significant (${\chi }^2=1.35$, p = 0.246). We, therefore, read the pattern as broadly similar magnitudes of substitution across the two domains rather than evidence of sharp differentiation between them.

5.4. Robustness Checks

5.4.1. CEM Analyses.

Although there is no ex ante reason to expect that the timing of the introduction of GNMT or ChatGPT was anticipated by market participants, some pretreatment coefficients in the event study specifications exhibit statistical noise. To further strengthen our identification strategy and reduce concerns about residual pretreatment imbalance, we implement CEM.

CEM constructs matched samples in which treated and control observations are identical on coarsened versions of selected covariates, thereby improving covariate balance and reducing model dependence. For both the GNMT and ChatGPT analyses, we match on rolling averages of transaction volumes, market size (measured one, three, and six months prior to treatment), pricing patterns, and the exact month of observation.

The estimates obtained from the matched samples are consistent with our baseline results as reported in Tables 4 and 5, reinforcing the robustness of our findings.

Table 4. CEM Sample Baseline Effects of GNMT and ChatGPT on Translation Markets

Table 5. CEM Sample Heterogeneous Effect of GNMT and ChatGPT on Different Translation Tasks: Regular Translations vs. Transcreations

5.4.2. Individual-Level Analysis.

We next examine changes in translation activity at the individual freelancer level following the introduction of GNMT and ChatGPT. The unit of analysis is the freelancer-month. For the GNMT analysis, we compare changes in transactions for freelancers active in treated language pairs prior to GNMT’s introduction with a counterfactual group of freelancers who did not specialize in those language pairs. We restrict the sample to freelancers who registered on the platform at least three months before the introduction of the first GNMT cohort. This restriction yields a conservative sample as freelancers with prior transaction history are more likely to secure work in a given period than newly registered sellers (Kokkodis and Ipeirotis 2016, Benson et al. 2020).

For the ChatGPT analysis, we impose the same three-month pretreatment registration requirement but adopt a balanced panel design. Specifically, we include only freelancers who completed at least one pure play translation task and at least one bundled translation task during the study period. This design allows each freelancer to serve as the freelancer’s own control when assessing the impact of generative AI.

Table 6 reports the individual-level results. For GNMT, we observe an 8.6% decline in freelancer transactions associated with treated language pairs following GNMT’s introduction (coefficient = −0.0903, p < 0.05). This pattern is consistent with the market-level estimates, indicating that aggregate reductions in translation activity are reflected in decreased demand for individual freelancers. For ChatGPT, we find a 9.1% reduction in translation transactions for pure play tasks relative to bundled translation tasks at the individual level (coefficient = −0.0949, p < 0.05).

Table 6. Individual-Level Estimates: GNMT vs. ChatGPT

The smaller magnitudes observed in the individual-level analysis relative to the market-level effects likely reflect the conservative nature of the sample construction. By focusing on established freelancers with prior platform experience, the analysis emphasizes workers who may be less exposed to substitution pressures than marginal or new entrants. Such freelancers may benefit from existing client relationships, accumulated reputation, or greater capacity to adjust their work practices in response to new technologies.

Event study estimates shown in Figure 6 illustrate the dynamic evolution of these effects. In both the GNMT and ChatGPT settings, treatment effects emerge shortly after technology introduction and persist throughout the observation window.

5.4.3. Other Robustness Checks.

We conduct a series of additional robustness checks to assess the sensitivity of our findings to alternative specifications, identification strategies, and potential confounds.

First, we test an alternative treatment timing for the Chinese–English language pair. Public announcements from Google’s Brain Team indicate that the first deployment of GNMT occurred for Chinese–English translation in September 2016 prior to the broader rollout. Reestimating our models using this earlier treatment date yields results consistent with the baseline estimates (Online Tables A4-1 through A4-3).

Second, because the GNMT rollout involved staggered treatment timing across language pairs, we address recent concerns about bias in two-way fixed effects estimators under treatment effect heterogeneity (Goodman-Bacon 2021, Baker et al. 2022, de Chaisemartin and D’Haultfoeuille 2026). We reestimate the GNMT effects using Wooldridge’s (2023) extended difference-in-differences estimator, which treats each treatment cohort separately and avoids problematic comparisons between early and late treated units. This approach is asymptotically equivalent to the group time average treatment effects framework of Callaway and Sant’Anna (2021) and accommodates the Poisson pseudo-maximum-likelihood specification required by the prevalence of zero outcomes in our data. The resulting estimates are qualitatively similar to those from our baseline models (Online Tables A5-1 through A5-3).

Third, in addition to transaction volume and revenue, we examine the effects of GNMT and ChatGPT on average prices. As reported in Online Table A6-1, we do not detect statistically significant changes in average prices for either technology. This pattern is consistent with prior evidence that freelancers in online labor markets tend to adjust quantities before prices in response to demand shocks (Cullen and Farronato 2021).

Fourth, we estimate an alternative panel differences specification for the ChatGPT analysis, comparing outcomes in the post-ChatGPT period (2022–2023) with a pre-COVID baseline period (2018–2019). This approach follows recent work using long-difference designs in related contexts (Eichenbaum et al. 2024, Goldberg et al. 2024). The estimates remain consistent with our main findings (Online Tables A7-1 through A7-3; Online Figure A7-1, (a) and (b)).

Fifth, we conduct placebo tests by assigning a false treatment date six months prior to the actual introduction of GNMT and ChatGPT and reestimating the models using only pretreatment data. These placebo analyses yield no statistically significant effects, supporting the validity of our identification strategy (Online Table A8-1).

Sixth, because monthly transaction data are inherently noisy, particularly for smaller language pairs with infrequent but high-variance demand, we aggregate the data to the quarterly level and reestimate the event study specifications. This substantially reduces measurement noise, and the resulting estimates are consistent with the monthly results (Online Tables A12-1 and A12-2, Online Figures A12-1 and A12-2).

Seventh, to address the large number of zero observations in the revenue data, we reestimate revenue effects using a Cragg’s hurdle (“churdle”) model. The first stage models the probability of observing positive revenue, and the second stage models revenue conditional on being positive using an inverse hyperbolic sine transformation. The results confirm our baseline findings: ChatGPT significantly reduces revenue for both regular and technical translations, whereas GNMT’s effects are smaller and less precisely estimated (Online Tables A9-1 and A9-2).

Eighth, to assess sensitivity to task classification, we reestimate our main specifications using an independent classifier based on bidirectional encoder representations from transformers (BERT).¹⁰ The results are qualitatively identical to the baseline classification: GNMT significantly reduces demand for regular translations but has no significant effect on transcreation (Online Table A15-1).

Ninth, to ensure comparability across our two research designs, we replicate the bundled task identification strategy used in the ChatGPT analysis on the GNMT sample. Specifically, we stack translation tasks (treatment) and performance tasks (control) within the same regression with language pair × task type fixed effects, mirroring the ChatGPT specification. The results are consistent with our primary GNMT estimates (Online Tables A13-1 and A13-2, Online Figure A13-1).

Tenth, we address the potential confound that Microsoft introduced neural machine translation for several major language pairs in November 2016. Excluding language pairs overlapping with Microsoft’s rollout yields no significant GNMT effect on total or regular translation. A triple-difference specification that explicitly interacts GNMT exposure with Microsoft overlap shows that the substitution effect is concentrated in overlapping language pairs (Online Table A14-1), indicating that increased availability of high-quality neural machine translation in major commercial markets amplified substitution effects.

Finally, to assess representativeness with respect to translation demand more broadly, we replicate our analysis using Google Trends data. For each treated and control language pair, we collect search query volumes for language pair–specific Google Translate terms over the January 2016–May 2017 period. Consistent with increased adoption of machine translation following GNMT’s introduction, search intensity rises for treated language pairs after treatment (Online Table A10-1, Online Figure A10-1).

6. Discussion and Conclusion

This study advances our understanding of AI’s substitution effects on human labor by providing task-level empirical evidence from two natural experiments in the domain of language translation. Exploiting the staggered introduction of a predictive AI system (GNMT) and the later introduction of a generative AI system (ChatGPT), we show that both technologies substitute for nonroutine cognitive work but in systematically different ways. Whereas GNMT primarily reduced demand for analytical tasks (i.e., regular translations), ChatGPT reduced demand for both analytical and intuitive tasks (i.e., transcreations) at comparable magnitudes. These findings highlight that the substitution effects of AI are heterogeneous across tasks and depend critically on the underlying architecture of the technology being deployed (Autor 2014, Frank et al. 2019, Choudhury et al. 2020, Acemoglu et al. 2022).

Importantly, the magnitude of these effects is economically meaningful. GNMT reduced demand for regular translation by approximately 9% within seven months of introduction, having no detectable effect on transcreation. By contrast, ChatGPT is associated with declines of more than 20% for regular translation and nearly 30% for transcreation over a comparable horizon. This shift is not merely qualitative (i.e., affecting more task types) but quantitative in terms of both effect size and in the compression of task-level heterogeneity. Under predictive AI, substitution effects were sharply differentiated across tasks; under generative AI, declines are large and broadly similar across analytical and intuitive work. This convergence in magnitudes suggests a fundamental change in the structure of task-level substitution rather than an incremental extension of earlier automation trends.

A central insight of our analysis is that the impact of ChatGPT should be interpreted as the marginal effect of generative AI in a labor market already shaped by six years of exposure to predictive AI. By 2022, predictive AI had already captured much of the analytical component of translation work. The additional reductions observed following ChatGPT’s introduction, particularly in transcreation tasks, therefore reflect an expansion of the scope of AI substitution rather than a reallocation within previously automated tasks. This sequential comparison allows us to isolate what is genuinely new about generative AI relative to earlier automation waves. Our goal is, therefore, not to compare the absolute magnitudes of GNMT and ChatGPT effects as if they were independent shocks to an unaltered market but to use the sequential structure to identify how the boundaries of AI substitution evolved as the underlying technology shifted from predictive to generative architectures.

Our findings lend empirical support to the distinction between analytical convergent and intuitive divergent tasks. GNMT, which is optimized for convergent sequence-to-sequence prediction, substitutes effectively for analytical translation but leaves intuitive transcreation largely unaffected. By contrast, ChatGPT’s generative architecture enables substitution across both task types. Importantly, this does not imply that generative AI perfectly replicates human intuition or cultural understanding; rather, it indicates that the level of performance required for substitution in market settings may be lower than the level required for parity on controlled benchmarks. As recent work on multilingual language models also suggests, generative systems can still falter when meaning hinges on subtle cultural or emotional cues (NLLB Team 2024).

This distinction helps reconcile our findings with evidence from benchmark-based and experimental studies. Whereas specialized models may outperform generative models on narrowly defined analytical benchmarks, our results show that a general purpose system such as ChatGPT can generate economically meaningful substitution effects for intuitive work in real labor markets. This suggests that task substitution in applied, interactive, and client-facing contexts may operate differently than performance in controlled evaluations.

Our results complement recent aggregate labor market evidence. Frey and Llanos-Paredes (2025), for example, document reduced translator employment growth following the diffusion of Google Translate with an instrumental variable estimate indicating a 0.84 percentage point reduction in employment growth per unit of adoption and an estimated 28,000 fewer positions created over the 2010–2023 period. Our task-level evidence reveals the mechanism underlying this aggregate pattern: predictive AI substitutes for analytical translation tasks, leaving intuitive work largely intact. This distinction is not observable in occupation-level data but becomes visible when labor demand is examined at the task level.

6.1. Managerial Implications

Beyond their theoretical contribution, our findings have direct implications for managerial decision making. For much of the past decade, a dominant managerial heuristic, rooted in experience with predictive AI, held that automation primarily threatens analytical work, whereas intuitive or creative tasks provide a safe harbor for human labor (Brynjolfsson and McAfee 2014, Frey and Osborne 2017). Our results show that this heuristic no longer generalizes in the era of generative AI. In this sense, our comparative design serves a belief-updating function: it provides empirical evidence that the task boundaries inferred from the predictive AI era do not hold under generative AI rather than comparing alternative software tools. Importantly, this belief-updating insight is not merely retrospective but forward looking. Specifically, if the empirical substitution boundary shifted once—from analytical tasks under predictive AI to both analytical and intuitive tasks under generative AI—managers should anticipate that it may shift again as AI systems continue to evolve. Therefore, processes such as workforce planning, reskilling, and task design frameworks built on a single generation of AI capabilities risk being overtaken by the next. Given this, the longitudinal lens we adopt here is best read as a template for continuous recalibration rather than a one-off diagnosis.

For managers, the key implication is that automation risk can no longer be assessed solely based on whether a task is creative or analytical. Instead, managers must evaluate the specific capabilities of the AI systems entering their domain and how those capabilities map onto the tasks embedded within jobs. Strategies that rely on reallocating workers from analytical tasks into creative roles as a hedge against automation may be effective under predictive AI but substantially less reliable under generative AI.

This insight has practical consequences for workforce planning, task redesign, and reskilling initiatives. Rather than organizing work around a simple human-versus-machine division, organizations may need to adopt more granular task-level strategies that identify where humans add value relative to rapidly evolving AI capabilities. In the translation context, for example, the growth of postediting services for both regular translation and transcreation suggests a shift toward hybrid roles that combine domain expertise with AI supervision and quality control (International Organization for Standardization 2017).

More broadly, our findings suggest that managerial responses to AI should emphasize organizational flexibility rather than task insulation. As AI systems evolve, previously reliable boundaries between human and machine work may erode. Managers who anchor their strategies to fixed notions of human-only tasks risk being repeatedly surprised as AI capabilities expand.

For individual workers, our results imply that specialization in tasks previously considered automation-resistant may no longer provide durable protection. Competitive advantage may instead arise from complementary skills such as domain knowledge, judgment, oversight, or integration that allow workers to work alongside AI systems rather than compete directly with them (Argyres and Zenger 2012, Teodoridis 2018).

At a broader level, our study contributes to ongoing debates about whether AI primarily substitutes for or augments human labor. By showing that generative AI can substitute for both analytical and intuitive tasks in real labor markets, we demonstrate that augmentation and substitution are not mutually exclusive outcomes but may coexist and evolve over time as technologies mature. More broadly, the task-level substitution effects we document in written translation may represent an early stage of a wider transformation in human communication. As AI-powered translation technologies extend from text-based platforms to real-time speech interfaces, such as AI translation–powered wearable devices capable of instantaneous verbal translation, similar substitution dynamics may emerge in domains well beyond formal labor markets. This trajectory raises important open questions for future research: whether widespread AI-mediated translation reduces incentives to learn foreign languages, whether it reshapes patterns of linguistic convergence or divergence, and whether the current global trend toward English as a dominant lingua franca may weaken if any language can be instantaneously translated using AI. Whereas these questions lie outside the scope of our data, our findings provide a task-level empirical foundation for understanding how advances in AI translation may reshape not only professional labor markets but also broader patterns of cross-cultural communication.

In sum, by comparing two generations of AI within the same labor market, our study provides a longitudinal perspective on how the boundaries of machine substitution shift as AI architectures evolve. This perspective clarifies why conclusions drawn from the predictive AI era do not straightforwardly generalize to the generative AI era and why managers, workers, and scholars alike must continually update their assumptions about the future of knowledge work.

6.2. Limitations

Our study has several limitations that point to opportunities for future research. First, our empirical context—language translation on a major OLM—has characteristics that may shape the observed effects. Freelance knowledge workers on OLMs differ from traditional employees in their contractual arrangements, pricing mechanisms, and exposure to market competition. Likewise, translation is particularly well suited to our identification strategy because of its clear task demarcations and discrete technological shocks. Whereas this context enables precise task-level analysis, the magnitude and timing of substitution effects may differ in other forms of knowledge work.

Second, our analysis focuses on language-based AI technologies (GNMT and ChatGPT). Although language-related AI has advanced rapidly, other domains such as speech recognition, computer vision, or robotics may follow different substitution trajectories. Extending a comparative, task-level approach to these domains is an important direction for future research.

Third, our estimates capture short- to medium-run effects following each technological introduction. Our decision to focus on a relatively contained posttreatment window was driven primarily by concerns about internal validity: extending the analysis horizon substantially increases exposure to confounding forces that are difficult to isolate from the treatment effect. Our event study specifications show no evidence of attenuation within the observation window, suggesting persistent rather than transient effects. Nevertheless, as AI models continue to evolve, training data expand, and complementary organizational practices adapt, longer run substitution and adjustment dynamics may differ from those documented here. Our contribution is, therefore, to establish clean task-level baselines for the predictive and generative AI eras against which longer run analyses can be compared.

Fourth, whereas we distinguish between regular (analytical) translation and transcreation (intuitive) tasks, this boundary is not always sharp in practice. Some translation projects combine literal conversion with creative adaptation. However, any resulting misclassification is unlikely to vary systematically across treatment and control groups or over time. Moreover, classification error in the dependent variable induces attenuation bias, shrinking estimated effects toward zero rather than exaggerating them. Correcting for symmetric misclassification rates of 7%–13% yields larger implied effects than those reported (Online Appendix A11). In addition, misclassification is asymmetric—transcreation tasks are more likely to be misclassified as regular than vice versa—biasing estimates toward finding effects on transcreation. Our null results for GNMT on transcreation are, therefore, conservative.

Fifth, translators often work across multiple language pairs. If freelancers use machine translation tools for supported language pairs as an intermediate input when completing work in unsupported pairs, spillovers to the control group could arise. Consistent with this view, approximately 14.3% of freelancers transact in both treated and control language pairs during the GNMT window. We acknowledge this as a limitation but note that any such spillover mechanism would operate similarly across task types within the same language pairs and, therefore, cannot explain the differential effects we observe between analytical and intuitive tasks.

Sixth, ChatGPT received substantially greater public attention than GNMT, which was introduced as a backend improvement to an existing product. Differences in awareness and adoption speed may, therefore, contribute to the larger observed effects for ChatGPT. From a managerial perspective, however, adoption dynamics are an integral component of a technology’s market impact. Our event study evidence indicates that effects emerge following actual availability rather than prelaunch media coverage. Our objective is to estimate the realized impact of AI technologies as they enter and diffuse into labor markets, which is precisely the quantity relevant for managerial decision making.

Finally, because the GNMT (2016–2017) and ChatGPT (2022–2023) shocks occurred in different macroeconomic environments, cross-technology comparisons should be interpreted as suggestive rather than definitive evidence of architectural differences. Relatedly, because Microsoft introduced neural machine translation for several overlapping language pairs in November 2016, the estimated GNMT effects for those pairs capture the joint impact of increased availability of high-quality neural machine translation rather than the effect of Google’s system alone (see Online Appendix A14).

Taken together, these limitations do not undermine our core contribution. Rather, they clarify the scope of our claims and highlight the value of task-level, longitudinal analyses for understanding how advances in AI reshape the boundaries of substitution for knowledge work.