Beyond Black Boxes: Designing and Testing Agentic AI Systems for Strategy

References

• Abou Ali M, Dornaika F, Charafeddine J (2025) Agentic AI: A comprehensive survey of architectures, applications, and future directions. Artificial Intelligence Rev. 59(1):11.Crossref, Google Scholar
• Agrawal A, Gans J, Goldfarb A (2022) Prediction Machines, Updated and Expanded: The Simple Economics of Artificial Intelligence (Harvard Business Review Press, Boston).Google Scholar
• Agrawal A, Camuffo A, Gans JS, Scott E, Stern S (2025) The Foundations of Bayesian Entrepreneurship (MIT Press, Cambridge, MA).Google Scholar
• Anthropic (2024a) Building effective agents. Accessed November 29, 2025, https://www.anthropic.com/research/building-effective-agents.Google Scholar
• Anthropic (2024b) Claude’s extended thinking. Accessed November 29, 2025, https://www.anthropic.com/news/visible-extended-thinking.Google Scholar
• Anthropic (2025) Introducing Claude 4. Accessed November 29, 2025, https://www.anthropic.com/news/claude-4.Google Scholar
• Baldwin C, MacCormack A, Rusnak J (2014) Hidden structure: Using network methods to map system architecture. Res. Policy 43(8):1381–1397.Crossref, Google Scholar
• Boussioux L, Lane JN, Zhang M, Jacimovic V, Lakhani KR (2024) The crowdless future? Generative AI and creative problem-solving. Organ. Sci. 35(5):1589–1607.Link, Google Scholar
• Cabigiosu A, Camuffo A (2016) Measuring modularity: Engineering and management effects of different approaches. IEEE Trans. Engrg. Management 64(1):103–114.Crossref, Google Scholar
• Camuffo A, Gambardella A, Pignataro A (2024) Theory-driven strategic management decisions. Strategy Sci. 9(4):382–396.Link, Google Scholar
• Chakraborty S, Pourreza M, Sun R, Song Y, Scherrer N, Huang F, Bedi AS, et al. (2025) On the role of feedback in test-time scaling of agentic AI workflows. Preprint, submitted April 2, https://arxiv.org/abs/2504.01931.Google Scholar
• Chalmers D, MacKenzie NG, Carter S (2021) Artificial intelligence and entrepreneurship: Implications for venture creation in the fourth industrial revolution. Entrepreneurship Theory Practice 45(5):1028–1053.Crossref, Google Scholar
• Chaturvedi A, Dasgupta M, Yadav N (2025) Tech-driven transformation: Unravelling the role of artificial intelligence in shaping strategic decision-making. Internat. J. Human-Comput. Interaction 41(19):12305–12324.Crossref, Google Scholar
• Cockburn IM, Henderson R, Stern S (2018) The impact of artificial intelligence on innovation: An exploratory analysis. Agrawal A, Gans J, Goldfarb A, eds. The Economics of Artificial Intelligence: An Agenda (University of Chicago Press, Chicago), 115–146.Google Scholar
• Conti A, Messinese D (2024) The selective tailwind effect of A.I. on startups: Predictions and anomalies. Preprint, submitted October 24, https://doi.org/10.2139/ssrn.4958898.Google Scholar
• Csaszar FA, Steinberger T (2022) Organizations as artificial intelligences: The use of artificial intelligence analogies in organization theory. Acad. Management Ann. 16(1):1–37.Crossref, Google Scholar
• Csaszar FA, Hinrichs N, Heshmati M (2024a) External representations in strategic decision-making: Understanding strategy’s reliance on visuals. Strategic Management J. 45(11):2191–2226.Crossref, Google Scholar
• Csaszar FA, Ketkar H, Kim H (2024b) Artificial intelligence and strategic decision-making: Evidence from entrepreneurs and investors. Strategy Sci. 9(4):322–345.Link, Google Scholar
• Dehghani N, Levin M (2024) Bio-inspired AI: Integrating biological complexity into artificial intelligence. Preprint, submitted November 22, https://arxiv.org/abs/2411.15243.Google Scholar
• Deng Z, Deng Z (2025) Becoming a cognitive miser? Antecedents and consequences of addictive ChatGPT use. Soc. Sci. Medicine 383:118467.Crossref, Google Scholar
• Derouiche H, Brahmi Z, Mazeni H (2025) Agentic AI frameworks: Architectures, protocols, and design challenges. Preprint, submitted August 13, https://arxiv.org/abs/2508.10146.Google Scholar
• Dietvorst BJ, Simmons JP, Massey C (2015) Algorithm aversion: People erroneously avoid algorithms after seeing them err. J. Experiment. Psych.: General 144(1):114–126.Crossref, Google Scholar
• Dietvorst BJ, Simmons JP, Massey C (2018) Overcoming algorithm aversion: People will use imperfect algorithms if they can (even slightly) modify them. Management Sci. 64(3):1155–1170.Link, Google Scholar
• Doshi AR, Bell JJ, Mirzayev E, Vanneste BS (2025) Generative artificial intelligence and evaluating strategic decisions. Strategic Management J. 46(3):583–610.Crossref, Google Scholar
• Ehrig T, Schmidt J (2022) Theory-based learning and experimentation: How strategists can systematically generate knowledge at the edge between the known and the unknown. Strategic Management J. 43(7):1287–1318.Crossref, Google Scholar
• Ethiraj SK, Levinthal D (2004) Bounded rationality and the search for organizational architecture: An evolutionary perspective on the design of organizations and their evolvability. Admin. Sci. Quart. 49(3):404–437.Crossref, Google Scholar
• Felin T, Holweg M (2024) Theory is all you need: AI, human cognition, and causal reasoning. Strategy Sci. 9(4):346–371.Link, Google Scholar
• Felin T, Zenger TR (2009) Entrepreneurs as theorists: On the origins of collective beliefs and novel strategies. Strategic Entrepreneurship J. 3(2):127–146.Crossref, Google Scholar
• Felin T, Zenger TR (2017) The theory-based view: Economic actors as theorists. Strategy Sci. 2(4):258–271.Link, Google Scholar
• Finio M, Downie A (2025) What is AI agent orchestration? Accessed November 29, 2025, https://www.ibm.com/think/topics/ai-agent-orchestration.Google Scholar
• Fiske STT, Taylor SE (2020) Social Cognition: From Brains to Culture (SAGE Publications Ltd., London).Google Scholar
• Gaessler F, Piezunka H (2023) Training with AI: Evidence from chess computers. Strategic Management J. 44(11):2724–2750.Crossref, Google Scholar
• Gans JS (2023) Experimental choice and disruptive technologies. Management Sci. 69(11):7044–7058.Link, Google Scholar
• Gans JS (2025) AI as strategist. NBER Working Paper No. 33650, National Bureau of Economic Research, Cambridge, MA.Google Scholar
• Gerber AS, Green DP (2017) Field experiments on voter mobilization: An overview of a burgeoning literature. Banerjee AV, Duflo E, eds. Handbook of Economic Field Experiments, vol. 1 (North Holland, Amsterdam), 395–438. Google Scholar
• Gurzick D, Joshi MP, Gurzick M, Csaszar F, Jia N (2026) AI-augmented strategic tools for strategy formulation and implementation: Revisiting traditional strategy tools & frameworks. Csaszar FA, Nick AM, Jia N, eds. Handbook of Artificial Intelligence and Strategy (Edward Elgar, Cheltenham, UK), 47–81.Google Scholar
• Harbarth L, Göβwein E, Bodemer D, Schnaubert L (2025) (Over)trusting AI recommendations: How system and person variables affect dimensions of complacency. Internat. J. Human-Comput. Interaction 41(1):391–410.Crossref, Google Scholar
• He G, Hemmer P, Vössing M, Schemmer M, Gadiraju U (2025) Fine-grained appropriate reliance: Human-AI collaboration with a multi-step transparent decision workflow for complex task decomposition. Preprint, submitted January 19, https://arxiv.org/abs/2501.10909.Google Scholar
• Heshmati M, Csaszar FA (2024) Learning strategic representations: Exploring the effects of taking a strategy course. Organ. Sci. 35(2):453–473.Link, Google Scholar
• Horowitz M, Kahn L, Macdonald J, Schneider J (2024) Adopting AI: How familiarity breeds both trust and contempt. Internat. J. Human-Comput. Interaction 41(1):1721–1735.Google Scholar
• Hünermund P, Kaminski J, Schmitt C (2021) Causal machine learning and business decision making. Preprint, submitted June 24, http://dx.doi.org/10.2139/ssrn.3867326.Google Scholar
• Kemp A (2024) Competitive advantage through artificial intelligence: Toward a theory of situated AI. Acad. Management Rev. 49(3):618–635.Crossref, Google Scholar
• Krakowski S, Luger J, Raisch S (2023) Artificial intelligence and the changing sources of competitive advantage. Strategic Management J. 44(6):1425–1452.Crossref, Google Scholar
• Microsoft (2025) AI agent orchestration patterns. Accessed November 29, 2025, https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/ai-agent-design-patterns.Google Scholar
• Miehling E, Ramamurthy KN, Varshney KR, Riemer M, Bouneffouf D, Richards JT, Dhurandhar A, et al. (2025) Agentic AI needs a systems theory. Preprint, submitted February 28, https://arxiv.org/abs/2503.00237.Google Scholar
• Miller CC, Chattopadhyay P, Bamberger P, Rockmann K (2025) Leveraging empirical abduction to bridge the rigor–Relevance divide: Celebrating 10 years of academy of management discoveries. Acad. Management Discoveries 11(3):325–331.Crossref, Google Scholar
• Obschonka M, Audretsch DB (2020) Artificial intelligence and big data in entrepreneurship: A new era has begun. Small Bus. Econom. 55:529–539.Crossref, Google Scholar
• Parasuraman R, Manzey DH (2010) Complacency and bias in human use of automation: An attentional integration. Human Factors 52(3):381–410.Crossref, Google Scholar
• Raisch S, Krakowski S (2021) Artificial intelligence and management: The automation–augmentation paradox. Acad. Management Rev. 46(1):192–210.Crossref, Google Scholar
• Ramoglou S, Chandra Y, Jin Q (2025) Opportunity search in the era of GenAI: Navigating uncertainty in an expanding universe of imaginable but unknowable futures. J. Management Stud., ePub ahead of print November 6, https://doi.org/10.1111/joms.70011.Google Scholar
• Romeo G, Conti D (2025) Exploring automation bias in human–AI collaboration: A review and implications for explainable AI. AI Soc. 41:259–278.Google Scholar
• Russell SJ, Norvig P (2016) Artificial Intelligence: A Modern Approach (Pearson Education, Hoboken, NJ).Google Scholar
• Sætre AS, Van de Ven A (2021) Generating theory by abduction. Acad. Management Rev. 46(4):684–701.Crossref, Google Scholar
• Sako M, Felin T (2025) Does AI prediction scale to decision making? Comm. ACM 68(4):18–21.Crossref, Google Scholar
• Sapkota R, Roumeliotis KI, Karkee M (2025) AI agents vs. agentic AI: A conceptual taxonomy, applications and challenges. Inform. Fusion 126:103599.Crossref, Google Scholar
• Sempf A, Hooker A (2025) Agentic AI patterns and workflows on AWS. Accessed November 29, 2025, https://docs.aws.amazon.com/prescriptive-guidance/latest/agentic-ai-patterns/introduction.html.Google Scholar
• Shavit Y, Agarwal S, Brundage M, Adler S, O’Keefe C, Campbell R, Lee T, et al. (2023) Practices for governing agentic AI systems. Research paper, OpenAI, San Francisco.Google Scholar
• Shepherd DA, Majchrzak A (2022) Machines augmenting entrepreneurs: Opportunities (and threats) at the nexus of artificial intelligence and entrepreneurship. J. Bus. Venturing 37(4):106227.Crossref, Google Scholar
• Simon HA (1962) The architecture of complexity. Proc. Amer. Philos. Soc. 106(6):467–482.Google Scholar
• Sorenson O (2024) Theory, search, and learning. Strategy Sci. 9(4):372–381.Link, Google Scholar
• Townsend DM, Hunt RA (2019) Entrepreneurial action, creativity, & judgment in the age of artificial intelligence. J. Bus. Venturing Insights 11:e00126.Crossref, Google Scholar
• Townsend DM, Hunt RA, Rady J, Manocha P, Hyeong JJ (2025) Are the futures computable? Knightian uncertainty and artificial intelligence. Acad. Management Rev. 50(2):415–440.Crossref, Google Scholar
• Tran KT, Dao D, Nguyen MD, Pham QV, O’Sullivan B, Nguyen HD (2025) Multi-agent collaboration mechanisms: A survey of LLMs. Preprint, submitted January 10, https://arxiv.org/abs/2501.06322.Google Scholar
• Wooldridge M (2009) An Introduction to Multiagent Systems (John Wiley & Sons, Hoboken, NJ).Google Scholar