The Crowdless Future? Generative AI and Creative Problem-Solving

References

• Abernathy WJ, Rosenbloom RS (1969) Parallel strategies in development projects. Management Sci. 15(10):B-486–B-505.Link, Google Scholar
• Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, Aleman FL, Almeida D, et al. (2023) GPT-4 technical report. Preprint, submitted March 15, https://arxiv.org/abs/2303.08774.Google Scholar
• Agrawal A, Gans J, Goldfarb A (2018) Prediction Machines: The Simple Economics of Artificial Intelligence (Harvard Business Review Press, Boston).Google Scholar
• Allen R, Choudhury P (2022) Algorithm-augmented work and domain experience: The countervailing forces of ability and aversion. Organ. Sci. 33(1):149–169.Link, Google Scholar
• Amabile TM (1983) The social psychology of creativity: A componential conceptualization. J. Personality Soc. Psych. 45(2):357–376.Crossref, Google Scholar
• Anderson BR, Shah JH, Kreminski M (2024) Homogenization effects of large language models on human creative ideation. Preprint, submitted February 2, https://arxiv.org/abs/2402.01536.Google Scholar
• Anthony C, Bechky BA, Fayard AL (2023) “Collaborating” with AI: Taking a system view to explore the future of work. Organ. Sci. 34(5):1672–1694.Link, Google Scholar
• Ash E, Hansen S (2023) Text algorithms in economics. Annual Rev. Econom. 15:659–688.Crossref, Google Scholar
• Ayers JW, Poliak A, Dredze M, Leas EC, Zhu Z, Kelley JB, Faix DJ, Goodman AM, Longhurst CA, Hogarth M (2023) Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Internal Medicine 183(6):589–596.Crossref, Google Scholar
• Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. Preprint, submitted September 1, https://arxiv.org/abs/1409.Google Scholar
• Bai Y, Kadavath S, Kundu S, Askell A, Kernion J, Jones A, Chen A, et al. (2022) Constitutional AI: Harmlessness from AI feedback. Preprint, submitted December 15, https://arxiv.org/abs/2212.08073.Google Scholar
• Barr PS, Stimpert JL, Huff AS (1992) Cognitive change, strategic action, and organizational renewal. Strategic Management J. 13(S1):15–36.Crossref, Google Scholar
• Baumol WJ (1993) Formal entrepreneurship theory in economics: Existence and bounds. J. Bus. Venturing 8(3):197–210.Crossref, Google Scholar
• Battle R, Gollapudi T (2024) The unreasonable effectiveness of eccentric automatic prompts. Preprint, submitted February 9, https://arxiv.org/abs/2402.10949.Google Scholar
• Becker GS (1994) Human Capital: A Theoretical and Empirical Analysis, with Special Reference to Education, 3rd ed. (The University of Chicago Press, Chicago).Google Scholar
• Bell JJ, Pescher C, Tellis GJ, Füller J (2024) Can AI help in ideation? A theory-based model for idea screening in crowdsourcing contests. Marketing Sci. 43(1):54–72.Link, Google Scholar
• Bellemare-Pepin A, Lespinasse F, Thölke P, Harel Y, Mathewson K, Olson J, Bengio Y, Jerbi K (2024) Divergent creativity in humans and large language models. Preprint, submitted May 13, https://arxiv.org/abs/2405.13012.Google Scholar
• Benner MJ, Tushman ML (2003) Exploitation, exploration, and process management: The productivity dilemma revisited. Acad. Management Rev. 28(2):238–256.Crossref, Google Scholar
• Boudreau KJ, Lacetera N, Lakhani KR (2011) Incentives and problem uncertainty in innovation contests: An empirical analysis. Management Sci. 57(5):843–863.Link, Google Scholar
• Boudreau KJ, Guinan EC, Lakhani KR, Riedl C (2016) Looking across and looking beyond the knowledge frontier: Intellectual distance, novelty, and resource allocation in science. Management Sci. 62(10):2765–2783.Link, Google Scholar
• Brand J, Israeli A, Ngwe D (2023) Using GPT for market research. Preprint, submitted March 30, https://doi.org/10.2139/ssrn.4395751.Google Scholar
• Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Neelakantan A, et al. (2020) Language models are few-shot learners. Preprint, submitted May 28, https://arxiv.org/abs/2005.14165.Google Scholar
• Brynjolfsson E, Li D, Raymond LR (2023) Generative AI at work. NBER Working Paper No. 31161, National Bureau of Economic Research, Cambridge, MA.Crossref, Google Scholar
• Bubeck S, Chandrasekaran V, Eldan R, Gehrke J, Horvitz E, Kamar E, Lee P, et al. (2023) Sparks of artificial general intelligence: Early experiments with GPT-4. Preprint, submitted March 22, https://arxiv.org/abs/2303.12712.Google Scholar
• Che YK, Gale I (2003) Optimal design of research contests. Amer. Econom. Rev. 93(3):646–671.Crossref, Google Scholar
• Chen M, Tworek J, Jun H, Yuan Q, Pinto HP de O, Kaplan J, Edwards H, et al. (2021) Evaluating large language models trained on code. Preprint, submitted July 7, https://arxiv.org/abs/2107.03374.Google Scholar
• Choudhury P, Allen RT, Endres MG (2021) Machine learning for pattern discovery in management research. Strategic Management J. 42(1):30–57.Crossref, Google Scholar
• Choudhary V, Marchetti A, Shrestha YR, Puranam P (2023) Human-AI ensembles: When can they work? J. Management, ePub ahead of print October 3, https://doi.org/10.1177/01492063231194968.Google Scholar
• Cyert RM, March JG (1963) A Behavioral Theory of the Firm (Prentice-Hall, Englewood Cliffs, NJ), 169–187.Google Scholar
• Dahan E, Mendelson H (2001) An extreme-value model of concept testing. Management Sci. 47(1):102–116.Link, Google Scholar
• DAIR.AI (2024) Prompt chaining. Prompting guide. Retrieved March 21, 2024, https://www.promptingguide.ai/techniques/prompt_chaining.Google Scholar
• Dell’Acqua F, McFowland E, Mollick ER, Lifshitz-Assaf H, Kellogg K, Rajendran S, Krayer L, Candelon F, Lakhani KR (2023) Navigating the jagged technological frontier: Field experimental evidence of the effects of AI on knowledge worker productivity and quality. Harvard Business School Working Paper No. 24-013, Harvard Business School, Boston.Google Scholar
• Dewar RD, Dutton JE (1986) The adoption of radical and incremental innovations: An empirical analysis. Management Sci. 32(11):1422–1433.Link, Google Scholar
• Doshi AR, Hauser O (2023) Generative artificial intelligence enhances creativity. Preprint, submitted August 14, https://doi.org/10.2139/ssrn.4535536.Google Scholar
• Felin T, Holweg M (2024) Theory is all you need: AI, human cognition, and decision making. Preprint, submitted April 4, https://doi.org/10.2139/ssrn.4737265.Google Scholar
• Fleming L, Sorenson O (2001) Technology as a complex adaptive system: Evidence from patent data. Res. Policy 30(7):1019–1039.Crossref, Google Scholar
• Fleming L, Mingo S, Chen D (2007) Collaborative brokerage, generative creativity, and creative success. Admin. Sci. Quart. 52(3):443–475.Crossref, Google Scholar
• Franceschelli G, Musolesi M (2023) On the creativity of large language models. Preprint, submitted March 27, https://arxiv.org/abs/2304.00008.Google Scholar
• Gavetti G, Levinthal D (2000) Looking forward and looking backward: Cognitive and experiential search. Admin. Sci. Quart. 45(1):113–137.Crossref, Google Scholar
• Gelman A, Hill J (2006) Data Analysis Using Regression and Multilevel/Hierarchical Models (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• Girotra K, Terwiesch C, Ulrich KT (2010) Idea generation and the quality of the best idea. Management Sci. 56(4):591–605.Link, Google Scholar
• Girotra K, Meincke L, Terwiesch C, Ulrich KT (2023) Ideas are dimes a dozen: Large language models for idea generation in innovation. Preprint, submitted August 2, https://doi.org/10.2139/ssrn.4526071.Google Scholar
• Glaeser EL, Laibson D, Sacerdote B (2002) An economic approach to social capital. Econom. J. 112(483):F437–F458.Google Scholar
• Gómez-Rodríguez C, Williams P (2023) A confederacy of models: A comprehensive evaluation of LLMs on creative writing. Preprint, submitted October 12, https://arxiv.org/abs/2310.08433.Google Scholar
• Guzik EE, Byrge C, Gilde C (2023) The originality of machines: AI takes the Torrance Test. J. Creativity 33(3):100065.Crossref, Google Scholar
• Hagendorff T, Fabi S, Kosinski M (2023) Human-like intuitive behavior and reasoning biases emerged in large language models but disappeared in ChatGPT. Nature Comput. Sci. 3(10):833–838.Crossref, Google Scholar
• Hargadon AB, Bechky BA (2006) When collections of creatives become creative collectives: A field study of problem solving at work. Organ. Sci. 17(4):484–500.Link, Google Scholar
• He VF, Shrestha YR, Puranam P, Miron-Spektor E (2023) Searching together: A theory of human-AI co-creativity. INSEAD Working Paper No. 2023/55/STR/OBH, INSEAD, Fontainebleau, France.Google Scholar
• Iansiti M, Lakhani KR (2020) Competing in the Age of AI: Strategy and Leadership When Algorithms and Networks Run the World (Harvard Business Review Press, Boston).Google Scholar
• Imbens GW, Rubin DB (2015) Causal Inference for Statistics, Social, and Biomedical Sciences (Cambridge University Press, New York).Crossref, Google Scholar
• Ivcevic Z, Grandinetti M (2024) Artificial intelligence as a tool for creativity. J. Creativity 34(2):10079.Crossref, Google Scholar
• Jang S (2017) Cultural brokerage and quality in multicultural teams. Organ. Sci. 28(6):993–1009.Link, Google Scholar
• Jeppesen LB, Lakhani KR (2010) Marginality and problem-solving effectiveness in broadcast search. Organ. Sci. 21(5):1016–1033.Link, Google Scholar
• Ji Z, Lee N, Frieske R, Yu T, Su D, Xu Y, Ishii E, Bang YJ, Madotto A, Fung P (2023) Survey of hallucination in natural language generation. ACM Comput. Surveys 55(12):1–38.Crossref, Google Scholar
• Jia N, Luo X, Fang Z, Liao C (2023) When and how artificial intelligence augments employee creativity. Acad. Management J. 67(1):5–32.Crossref, Google Scholar
• Kaplan S, Vakili K (2015) The double‐edged sword of recombination in breakthrough innovation. Strategic Management J. 36(10):1435–1457.Crossref, Google Scholar
• Katila R, Ahuja G (2002) Something old, something new: A longitudinal study of search behavior and new product introduction. Acad. Management J. 45(6):1183–1194.Crossref, Google Scholar
• Kenny D, Kashy D, Cook W, Simpson J (2006) Dyadic Data Analysis (The Guildford Press, New York).Google Scholar
• Kim H, Glaeser EL, Hillis A, Kominers SD, Luca M (2024) Decision authority and the returns to algorithms. Strategic Management J. 45(4):619–648.Crossref, Google Scholar
• Kleinberg J, Lakkaraju H, Leskovec J, Ludwig J, Mullainathan S (2018) Human decisions and machine predictions. Quart. J. Econom. 133(1):237–293.Crossref, Google Scholar
• Kneeland MK, Schilling MA, Aharonson BS (2020) Exploring uncharted territory: Knowledge search processes in the origination of outlier innovation. Organ. Sci. 31(3):535–557.Link, Google Scholar
• Koivisto M, Grassini S (2023) Best humans still outperform artificial intelligence in a creative divergent thinking task. Sci. Rep. 13(1):13601.Crossref, Google Scholar
• Kong A, Zhao S, Chen H, Li Q, Qin Y, Sun R, Zhou X (2023) Better zero-shot reasoning with role-play prompting. Preprint, submitted August 15, https://arxiv.org/abs/2308.07702.Google Scholar
• Kuznetsova A, Brockhoff PB, Christensen RHB (2017) lmerTest package: Tests in linear mixed effects models. J. Statist. Software 82(13):1–26.Crossref, Google Scholar
• Laursen K, Salter A (2006) Open for innovation: The role of openness in explaining innovation performance among UK manufacturing firms. Strategic Management J. 27(2):131–150.Crossref, Google Scholar
• Lebovitz S, Lifshitz-Assaf H, Levina N (2022) To engage or not to engage with AI for critical judgments: How professionals deal with opacity when using AI for medical diagnosis. Organ. Sci. 33(1):126–148.Link, Google Scholar
• Leiponen A, Helfat CE (2010) Innovation objectives, knowledge sources, and the benefits of breadth. Strategic Management J. 31(2):224–236.Crossref, Google Scholar
• Lenth R, Love J, Herve M (2018) emmeans: Estimated marginal means, aka least-squares means. Accessed May 20, 2024, https://cran.r-project.org/package=emmeans.Google Scholar
• Levine SS, Schilke O, Kacperczyk O, Zucker LG (2023) Primer for experimental methods in organization theory. Organ. Sci. 34(6):1997–2025.Link, Google Scholar
• Levinthal DA (1997) Adaptation on rugged landscapes. Management Sci. 43(7):934–950.Link, Google Scholar
• Lewis P, Perez E, Piktus A, Petroni F, Karpukhin V, Goyal N, Küttler H, et al. (2020) Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances Neural Inform. Processing Systems 33:9459–9474.Google Scholar
• Li D, Raymond LR, Bergman P (2020) Hiring as exploration. NBER Working Paper No. 27736, National Bureau of Economic Research, Cambridge, MA.Google Scholar
• Lifshitz-Assaf H (2018) Dismantling knowledge boundaries at NASA: The critical role of professional identity in open innovation. Admin. Sci. Quart. 63(4):746–782.Crossref, Google Scholar
• Lindebaum D, Vesa M, Den Hond F (2020) Insights from “the machine stops” to better understand rational assumptions in algorithmic decision making and its implications for organizations. Acad. Management Rev. 45(1):247–263.Crossref, Google Scholar
• Lou B, Wu L (2021) AI on drugs: Can artificial intelligence accelerate drug development? Evidence from a large-scale examination of bio-pharma firms. Management Inform Systems Quart. 45(3):1451–1482.Crossref, Google Scholar
• March JG (1991) Exploration and exploitation in organizational learning. Organ. Sci. 2(1):71–87.Link, Google Scholar
• Meincke L, Mollick ER, Terwiesch C (2024) Prompting diverse ideas: Increasing AI idea variance. Preprint, submitted January 27, https://arxiv.org/abs/2402.01727.Google Scholar
• Miric M, Jia N, Huang KG (2023) Using supervised machine learning for large‐scale classification in management research: The case for identifying artificial intelligence patents. Strategic Management J. 44(2):491–519.Crossref, Google Scholar
• Mollick E, Nanda R (2016) Wisdom or madness? Comparing crowds with expert evaluation in funding the arts. Management Sci. 62(6):1533–1553.Link, Google Scholar
• Nelson RR (1961) Uncertainty, learning, and the economics of parallel research and development efforts. Rev. Econom. Statist. 43(4):351–364.Crossref, Google Scholar
• Nelson RR, Winter SG (1982) The Schumpeterian tradeoff revisited. Amer. Econom. Rev. 72(1):114–132.Google Scholar
• Nickerson JA, Zenger TR (2004) A knowledge-based theory of the firm—The problem-solving perspective. Organ. Sci. 15(6):617–632.Link, Google Scholar
• Noy S, Zhang W (2023) Experimental evidence on the productivity effects of generative artificial intelligence. Preprint, submitted March 6, https://doi.org/10.2139/ssrn.4375283.Google Scholar
• Ocasio W (1997) Toward an attention‐based view of the firm. Strategic Management J. 18(S1):187–206.Crossref, Google Scholar
• OpenAI (2024) Strategy: Write clear instructions. Accessed July 11, 2024, https://platform.openai.com/docs/guides/prompt-engineering.Google Scholar
• Otis N, Clarke RP, Delecourt S, Holtz D, Koning R (2023) The uneven impact of generative AI on entrepreneurial performance. Preprint, submitted January 17, https://doi.org/10.2139/ssrn.4671369.Google Scholar
• Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C, Mishkin P, Zhang C, Agarwal S, Slama K, Ray A (2022) Training language models to follow instructions with human feedback. Advances Neural Inform. Processing Systems 35:27730–27744.Google Scholar
• Paik JH, Scholl M, Sergeev R, Randazzo S, Lakhani KR (2020) Innovation contests for high-tech procurement. Res. Tech. Management 63(2):36–45.Crossref, Google Scholar
• Perry-Smith JE (2006) Social yet creative: The role of social relationships in facilitating individual creativity. Acad. Management J. 49(1):85–101.Crossref, Google Scholar
• Perry-Smith JE, Mannucci PV (2017) From creativity to innovation: The social network drivers of the four phases of the idea journey. Acad. Management Rev. 42(1):53–79.Crossref, Google Scholar
• Piezunka H, Dahlander L (2015) Distant search, narrow attention: How crowding alters organizations’ filtering of suggestions in crowdsourcing. Acad. Management J. 58(3):856–880.Crossref, Google Scholar
• Piezunka H, Dahlander L (2019) Idea rejected, tie formed: Organizations’ feedback on crowdsourced ideas. Acad. Management J. 62(2):503–530.Crossref, Google Scholar
• Raisch S, Fomina K (2023) Combining human and artificial intelligence: Hybrid problem-solving in organizations. Acad. Management Rev., ePub ahead of print January 5, https://doi.org/10.5465/amr.2021.0421.Google Scholar
• Riedl C, Grad T, Lettl C (2024) Competition and collaboration in crowdsourcing communities: What happens when peers evaluate each other? Organ. Sci., ePub ahead of print April 30, https://doi.org/10.1287/orsc.2021.15163.Link, Google Scholar
• Renze M, Guven E (2024) The effect of sampling temperature on problem solving in large language models. Preprint, submitted February 7, https://arxiv.org/abs/2402.05201.Google Scholar
• Rhee L, Leonardi PM (2018) Which pathway to good ideas? An attention‐based view of innovation in social networks. Strategic Management J. 39(4):1188–1215.Crossref, Google Scholar
• Rindova VP, Petkova AP (2007) When is a new thing a good thing? Technological change, product form design, and perceptions of value for product innovations. Organ. Sci. 18(2):217–232.Link, Google Scholar
• Saravia E (2022) Prompt engineering guide. Accessed June 22, 2024, https://www.promptingguide.ai/.Google Scholar
• Shanahan M, McDonell K, Reynolds L (2023) Role play with large language models. Nature 623(7987):493–498.Crossref, Google Scholar
• Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529(7587):484–489.Crossref, Google Scholar
• Simon HA (1973) The structure of ill structured problems. Artificial Intelligence 4(3–4):181–201.Crossref, Google Scholar
• Stevenson C, Smal I, Baas M, Grasman R, van der Maas H (2022) Putting GPT-3′s creativity to the (alternative uses) test. Preprint, submitted June 10, https://arxiv.org/abs/2206.08932.Google Scholar
• Taylor CR (1995) Digging for golden carrots: An analysis of research tournaments. Amer. Econom. Rev. 85(4):872–890.Google Scholar
• Teodoridis F, Bikard M, Vakili K (2019) Creativity at the knowledge frontier: The impact of specialization in fast-and slow-paced domains. Admin. Sci. Quart. 64(4):894–927.Crossref, Google Scholar
• Terwiesch C, Ulrich KT (2009) Innovation Tournaments: Creating and Selecting Exceptional Opportunities (Harvard Business Review Press, Boston).Google Scholar
• Terwiesch C, Xu Y (2008) Innovation contests, open innovation, and multiagent problem solving. Management Sci. 54(9):1529–1543.Link, Google Scholar
• Tong S, Jia N, Luo X, Fang Z (2021) The Janus face of artificial intelligence feedback: Deployment vs. disclosure effects on employee performance. Strategic Management J. 42(9):1600–1631.Crossref, Google Scholar
• Tripsas M (2009) Technology, identity, and inertia through the lens of “The Digital Photography Company.” Organ. Sci. 20(2):441–460.Link, Google Scholar
• Van de Ven AH (1986) Central problems in the management of innovation. Management Sci. 32(5):590–607.Link, Google Scholar
• Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Advances Neural Inform. Processing Systems 28:30.Google Scholar
• Wang L, Ma C, Feng X, Zhang Z, Yang H, Zhang J, Chen Z, et al. (2024) A survey on large language model based autonomous agents. Front. Comput. Sci. 18(6):186345.Google Scholar
• Weber M (1978) Economy and Society: An Outline of Interpretive Sociology (University of California Press, Berkeley, CA).Google Scholar
• Wei J, Wang X, Schuurmans D, Bosma M, Ichter B, Xia F, Chi E, Le Q, Zhou D (2022) Chain-of-thought prompting elicits reasoning in large language models. Preprint, submitted January 28, https://arxiv.org/abs/2201.11903.Google Scholar
• Wuchty S, Jones BF, Uzzi B (2007) The increasing dominance of teams in production of knowledge. Science 316(5827):1036–1039.Crossref, Google Scholar
• Xi Z, Chen W, Guo X, He W, Ding Y, Hong B, Zhang M, Wang J, Jin S, Zhou E (2023) The rise and potential of large language model based agents: A survey. Preprint, submitted September 14, https://arxiv.org/abs/2309.07864.Google Scholar
• Yager KG (2023) Domain-specific chatbots for science using embeddings. Digital Discovery 2(6):1850–1861.Crossref, Google Scholar
• Yin Z, Wang H, Horio K, Kawahara D, Sekine S (2024) Should we respect LLMs? A cross-lingual study on the influence of prompt politeness on LLM performance. Preprint, submitted February 22, https://arxiv.org/abs/2402.14531.Google Scholar
• Yin S, Fu C, Zhao S, Li K, Sun X, Xu T, Chen E (2023) A survey on multimodal large language models. Preprint, submitted June 23, https://arxiv.org/abs/2306.13549.Google Scholar
• Zamfirescu-Pereira JD, Wong RY, Hartmann B, Yang Q (2023) Why Johnny can’t prompt: How non-AI experts try (and fail) to design LLM prompts. Proc. 2023 CHI Conf. Human Factors Comput. Systems (CHI ‘23) (Association for Computing Machinery, New York).Google Scholar