Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality

References

• Acemoglu D, Autor D (2011) Skills, tasks and technologies: Implications for employment and earnings. Card D, Ashenfelter O, eds. Handbook of Labor Economics, vol. 4 (Elsevier, Amsterdam), 1043–1171.Google Scholar
• Agrawal A, Gans J, Goldfarb A (2018) Prediction Machines: The Simple Economics of Artificial Intelligence (Harvard Business Press, Boston).Google Scholar
• Ali R, Tang OY, Connolly ID, Fridley JS, Shin JH, Zadnik Sullivan PL, Cielo D, et al. (2023) Performance of ChatGPT, GPT-4, and Google Bard on a neurosurgery oral boards preparation question bank. Neurosurgery 93(5):1090–1098.Crossref, Google Scholar
• Allen R, Choudhury P (2022) Algorithm-augmented work and domain experience: The countervailing forces of ability and aversion. Organ. Sci. 33(1):149–169.Link, Google Scholar
• Anthony C, Bechky BA, Fayard A-L (2023) “Collaborating” with AI: Taking a system view to explore the future of work. Organ. Sci. 34(5):1672–1694.Link, Google Scholar
• Aral S, Brynjolfsson E, Van Alstyne M (2012) Information, technology, and information worker productivity. Inform. Systems Res. 23(3 Part 2):849–867.Link, Google Scholar
• Athey S, Bryan K, Gans J (2020) The allocation of decision authority to human and artificial intelligence. AEA Papers Proc. 110(1):80–84.Crossref, Google Scholar
• Bailey DE, Faraj S, Hinds PJ, Leonardi PM, von Krogh G (2022) We are all theorists of technology now: A relational perspective on emerging technology and organizing. Organ. Sci. 33(1):1–18.Link, Google Scholar
• Barrett M, Oborn E, Orlikowski WJ, Yates J (2012) Reconfiguring boundary relations: Robotic innovations in pharmacy work. Organ. Sci. 23(5):1448–1466.Link, Google Scholar
• Beane M (2019) Shadow learning: Building robotic surgical skill when approved means fail. Admin. Sci. Quart. 64(1):87–123.Crossref, Google Scholar
• Beane MI, Leonardi PM (2025) Pace layering as a metaphor for organizing in the age of intelligent technologies: Considering the future of work by theorizing the future of organizing. J. Management Stud. 62(5):2025–2052.Crossref, Google Scholar
• Berg JM, Raj M, Seamans R (2023) Capturing value from artificial intelligence. Acad. Management Discoveries 9(4):424–428.Crossref, Google Scholar
• Blandin A, Bick A, Deming DJ (2026) The rapid adoption of generative AI. Management Sci., ePub ahead of print January 20, https://doi.org/10.1287/mnsc.2025.02523.Google Scholar
• Boiko DA, MacKnight R, Kline B, Gomes G (2023) Autonomous chemical research with large language models. Nature 624(7992):570–578.Crossref, Google Scholar
• Boussioux L, Lane JN, Zhang M, Jacimovic V, Lakhani KR (2024) The crowdless future? Generative AI and creative problem-solving. Organ. Sci. 35(5):1589–1607.Link, Google Scholar
• Brynjolfsson E, Jin W, McElheran K (2021) The power of prediction: Predictive analytics, workplace complements, and business performance. Bus. Econom. 56(4):217–239.Crossref, Google Scholar
• Brynjolfsson E, Li D, Raymond LR (2025) Generative AI at work. Quart. J. Econom. 140(2):889–942.Crossref, Google Scholar
• Brynjolfsson E, Mitchell T, Rock D (2018) What can machines learn and what does it mean for occupations and the economy? AEA Papers Proc. 108(1):43–47.Crossref, Google Scholar
• Buçinca Z, Malaya MB, Gajos KZ (2021) To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proc. ACM Human-Comput. Interaction 5(CSCW1):188.Crossref, Google Scholar
• Çalli E, Sogancioglu E, van Ginneken B, van Leeuwen KG, Murphy K (2021) Deep learning for chest X-ray analysis: A survey. Medical Image Anal. 73(2):102125.Crossref, Google Scholar
• Choi JH, Schwarcz D (2024) AI assistance in legal analysis: An empirical study. J. Legal Ed. 73(2):384–420.Google Scholar
• Choudhary V, Marchetti A, Shrestha YR, Puranam P (2023) Human-AI ensembles: When can they work? J. Management 51(2):536–569.Google Scholar
• Cowgill B, Dell’Acqua F, Deng S, Hsu D, Verma N, Chaintreau A (2020) Biased programmers? Or biased data? A field experiment in operationalizing AI ethics. Proc. 21st ACM Conf. Econom. Comput. (EC '20) (Association for Computing Machinery, New York), 679–681.Google Scholar
• Davies A, Veličković P, Buesing L, Blackwell S, Zheng D, Tomašev N, Tanburn R, et al. (2021) Advancing mathematics by guiding human intuition with AI. Nature 600(7887):70–74.Crossref, Google Scholar
• Dell’Acqua F (2022) Falling asleep at the wheel: Human/AI collaboration in a field experiment on HR recruiters. Working paper, Laboratory for Innovation Science, Harvard Business School, Boston.Google Scholar
• Dell’Acqua F, Kogut B, Perkowski P (2025) Super Mario meets AI: Experimental effects of automation and skills on team performance and coordination. Rev. Econom. Statist. 107(4):951–966.Crossref, Google Scholar
• DeStefano T, Kellogg K, Menietti M, Vendraminelli L (2022) Why providing humans with interpretable algorithms may, counterintuitively, lead to lower decision-making performance. MIT Sloan Research Paper No. 6797, Massachusetts Institute of Technology, Cambridge.Google Scholar
• Drucker PF (1959) Landmarks of Tomorrow: A Report on the New Post Modern World (Routledge, Abingdon-on-Thames, UK).Google Scholar
• Eloundou T, Manning S, Mishkin P, Rock D (2024) GPTs are GPTs: Labor market impact potential of LLMs. Science 384(6702):1306–1308.Crossref, Google Scholar
• Faraj S, Leonardi PM (2022) Strategic organization in the digital age: Rethinking the concept of technology. Strategic Organ. 20(4):771–785.Crossref, Google Scholar
• Felten EW, Raj M, Seamans R (2023) Occupational heterogeneity in exposure to generative AI. Preprint, submitted April 10, https://doi.org/10.2139/ssrn.4414065.Google Scholar
• Feuerriegel S, Shrestha YR, von Krogh G, Zhang C (2022) Bringing artificial intelligence to business management. Nature Machine Intelligence 4(7):611–613.Crossref, Google Scholar
• Furman J, Seamans R (2019) AI and the economy. Innovation Policy Econom. 19(1):161–191.Crossref, Google Scholar
• Gaessler F, Piezunka H (2023) Training with AI: Evidence from chess computers. Strategic Management J. 44(11):2724–2750.Crossref, Google Scholar
• Geerling W, Mateer GD, Wooten J, Damodaran N (2023) ChatGPT has aced the test of understanding in college economics: Now what? Amer. Economist 68(2):233–245.Crossref, Google Scholar
• Glaeser E, Hillis A, Kim H, Kominers SD, Luca M (2024) Decision authority and the returns to algorithms. Strategic Management J. 45(4):619–648.Crossref, Google Scholar
• Hui X, Reshef O, Zhou L (2024) The short-term effects of generative artificial intelligence on employment: Evidence from an online labor market. Organ. Sci. 35(6):1977–1989.Link, Google Scholar
• Humlum A, Vestergaard E (2025) The unequal adoption of ChatGPT exacerbates existing inequalities among workers. Proc. Natl. Acad. Sci. USA 122(1):e2414972121.Crossref, Google Scholar
• Iansiti M, Lakhani KR (2020) Competing in the Age of AI: Strategy and Leadership When Algorithms and Networks Run the World (Harvard Business Press, Boston).Google Scholar
• Karpathy A (2024) Commentary on jagged capabilities of large language models. Twitter (July 25, 2024), https://x.com/karpathy/status/1816531576228053133.Google Scholar
• Karpathy A (2025) 2025 LLM year in review. Accessed February 3, 2026, https://karpathy.bearblog.dev/year-in-review-2025/.Google Scholar
• Kellogg KC (2022) Local adaptation without work intensification: Experimentalist governance of digital technology for mutually beneficial role reconfiguration in organizations. Organ. Sci. 33(2):571–599.Link, Google Scholar
• Kellogg KC, Myers JE, Gainer L, Singer SJ (2021) Moving violations: Pairing an illegitimate learning hierarchy with trainee status mobility for acquiring new skills when traditional expertise erodes. Organ. Sci. 32(1):181–209.Link, Google Scholar
• Lebovitz S, Levina N, Lifshitz-Assaf H (2021) Is AI ground truth really true? The dangers of training and evaluating AI tools based on experts’ know-what. MIS Quart. 45(3):1501–1526.Crossref, Google Scholar
• Lebovitz S, Lifshitz-Assaf H, Levina N (2022) To engage or not to engage with AI for critical judgments: How professionals deal with opacity when using AI for medical diagnosis. Organ. Sci. 33(1):126–148.Link, Google Scholar
• Lee P, Bubeck S, Petro J (2023) Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine. New England J. Medicine 388(13):1233–1239.Crossref, Google Scholar
• Meincke L, Girotra K, Nave G, Terwiesch C, Ulrich KT (2024) Using large language models for idea generation in innovation. Working paper, Operations, Information and Decisions, The Wharton School, University of Pennsylvania, Philadelphia.Google Scholar
• Monisha R, Sen S, Davangeri RU, Sri Lakshmi KS, Dey S (2021) An approach toward design and implementation of distributed framework for astronomical big data processing. Udgata SK, Sethi S, Gao XZ, eds. Intelligent Systems, Lecture Notes in Networks and Systems, vol. 431 (Springer, Singapore), 267–275.Google Scholar
• Moor M, Banerjee O, Abad ZSH, Krumholz HM, Leskovec J, Topol EJ, Rajpurkar P (2023) Foundation models for generalist medical artificial intelligence. Nature 616(7956):259–265.Crossref, Google Scholar
• Noy S, Zhang W (2023) Experimental evidence on the productivity effects of generative artificial intelligence. Science 381(6654):187–192.Crossref, Google Scholar
• OpenAI (2023) GPT-4 technical report. Preprint, submitted March 15, https://arxiv.org/abs/2303.08774.Google Scholar
• Otis NG, Delecourt S, Cranney K, Koning R (2024a) Global evidence on gender gaps and generative AI. Working Paper No. 25-023, Harvard Business School, Boston.Google Scholar
• Otis N, Clarke R, Delecourt S, Holtz D, Koning R (2024b) The uneven impact of generative AI on entrepreneurial performance. Preprint, submitted February 27, https://doi.org/10.2139/ssrn.4671369.Google Scholar
• Peng S, Kalliamvakou E, Cihon P, Demirer M (2023) The impact of AI on developer productivity: Evidence from GitHub Copilot. Preprint, submitted February 13, https://arxiv.org/abs/2302.06590.Google Scholar
• Pichai S (2025) Interview with Lex Fridman. Lex Fridman Podcast #471. Transcript. Accessed February 3, 2026, https://lexfridman.com/sundar-pichai-transcript.Google Scholar
• Raisch S, Krakowski S (2021) Artificial intelligence and management: The automation–augmentation paradox. Acad. Management Rev. 46(1):192–210.Crossref, Google Scholar
• Randazzo S, Joshi A, Kellogg KC, Lifshitz H, Dell’Acqua F, Lakhani KR (2025a) GenAI as a power persuader: How professionals get persuasion bombed when they attempt to validate LLMs. Working Paper No. 26-021, Harvard Business School, Boston.Google Scholar
• Randazzo S, Lifshitz H, Kellogg KC, Dell Acqua F, Mollick E, Candelon F, Lakhani KR (2025b) Cyborgs, centaurs and self-automators: The three modes of human–GenAI knowledge work and their implications for skilling and the future of expertise. Working Paper No. 26-036, Harvard Business School, Boston.Google Scholar
• Reed S, Zolna K, Parisotto E, Gomez Colmenarejo S, Novikov A, Barth-Maron G, Gimenez M, et al. (2022) A generalist agent. Preprint, submitted November 11, https://arxiv.org/abs/2205.06175.Google Scholar
• Schaeffer R, Miranda B, Koyejo S (2023) Are emergent abilities of large language models a mirage? Adv. Neural Inform. Processing Systems, vol. 36 (Curran Associates Inc., Red Hook, NY), 55565–55581.Google Scholar
• Sergeeva AV, Faraj S, Huysman M (2020) Losing touch: An embodiment perspective on coordination in robotic surgery. Organ. Sci. 31(5):1248–1271.Link, Google Scholar
• Singhal K, Azizi S, Tu T, Mahdavi SS, Wei J, Chung HW, Scales N, et al. (2023) Large language models encode clinical knowledge. Nature 620(7972):172–180.Crossref, Google Scholar
• Skitka LJ, Mosier KL, Burdick M (1999) Does automation bias decision-making? Internat. J. Human-Comput. Stud. 51(5):991–1006.Crossref, Google Scholar
• Vaccaro M, Almaatouq A, Malone TW (2024) When combinations of humans and AI are useful: A systematic review and meta-analysis. Nature Human Behav. 8(12):2293–2303.Crossref, Google Scholar
• Vlamis K, Varanasi L, Paradis T (2024) MBB explained: How hard it is to get hired and what it’s like to work for the prestigious strategy consulting firms, McKinsey, Bain, and BCG. Bus. Insider Africa (November 29), https://africa.businessinsider.com/careers/mbb-explained-how-hard-it-is-to-get-hired-and-what-its-like-to-work-for-the/v6gf0lp.Google Scholar
• Zhou E, Lee D (2024) Generative artificial intelligence, human creativity, and art. PNAS Nexus 3(3):pgae052.Crossref, Google Scholar