Large Language Model in Creative Work: The Role of Collaboration Modality and User Expertise

References

• Agarwal A, Mukhopadhyay T (2016) The impact of competing ads on click performance in sponsored search. Inform. Systems Res. 27(3):538–557.Link, Google Scholar
• Akerman A, Gaarder I, Mogstad M (2015) The skill complementarity of broadband internet. Quart. J. Econom. 130(4):1781–1824.Crossref, Google Scholar
• Arnett DB, Wittmann CM (2014) Improving marketing success: The role of tacit knowledge exchange between sales and marketing. J. Bus. Res. 67(3):324–331.Crossref, Google Scholar
• Bauer K, Gill A (2024) Mirror, mirror on the wall: Algorithmic assessments, transparency, and self-fulfilling prophecies. Inform. Systems Res. 35(1):226–248.Link, Google Scholar
• Bauer K, von Zahn M, Hinz O (2023) Expl(AI)ned: The impact of explainable artificial intelligence on users’ information processing. Inform. Systems Res. 34(4):1582–1602.Link, Google Scholar
• Berg JM (2014) The primal mark: How the beginning shapes the end in the development of creative ideas. Organ. Behav. Human Decision Processes 125(1):1–17.Crossref, Google Scholar
• Boussioux L, Lane JN, Zhang M, Jacimovic V, Lakhani KR (2024) The crowdless future? Generative AI and creative problem solving. Org. Sci., ePub ahead of print August 16, https://doi.org/10.1287/orsc.2023.18430.Google Scholar
• Brand J, Israeli A, Ngwe D (2023) Using GPT for market research. Working paper, Harvard University, Cambridge, MA.Google Scholar
• Brandts J, Groenert V, Rott C (2015) The impact of advice on women’s and men’s selection into competition. Management Sci. 61(5):1018–1035.Link, Google Scholar
• Brynjolfsson E, Li D, Raymond LR (2023) Generative AI at work. Working paper, Standford University, Santa Clara, CA.Google Scholar
• Bubeck S, Chandrasekaran V, Eldan R, Gehrke J, Horvitz E, Kamar E, Lee P, et al. (2023) Sparks of artificial general intelligence: Early experiments with GPT-4. Working paper, Microsoft, Richmond, WA.Google Scholar
• Bunescu RC, Uduehi OO (2019) Learning to surprise: A composer-audience architecture. Internat. Conf. Innovative Comput. Cloud Comput. (ICCC’19) (Charlotte, NC), 41–48.Google Scholar
• Cho C-H, Cheon HJ (2004) Why do people avoid advertising on the internet? J. Advertising 33(4):89–97.Crossref, Google Scholar
• Darley WK, Smith RE (1993) Advertising claim objectivity: Antecedents and effects. J. Marketing 57(4):100–113.Crossref, Google Scholar
• Das G, Wiener HJD, Kareklas I (2019) To emoji or not to emoji? Examining the influence of emoji on consumer reactions to advertising. J. Bus. Res. 96:147–156.Crossref, Google Scholar
• Davenport TH, Mittal N (2022) How generative AI is changing creative work. Harvard Bus. Rev. (November 14), https://hbr.org/2022/11/how-generative-ai-is-changing-creative-work.Google Scholar
• Dell’Acqua F, McFowland E, Mollick ER, Lifshitz-Assaf H, Kellogg K, Rajendran S, Krayer L, Candelon F, Lakhani KR (2023) Navigating the jagged technological frontier: Field experimental evidence of the effects of AI on knowledge worker productivity and quality. Working paper, Havard Business School, Cambridge, MA.Google Scholar
• Di Fede G, Rocchesso D, Dow SP, Andolina S (2022) The idea machine: LLM-based expansion, rewriting, combination, and suggestion of ideas. Proc. 14th Conf. Creativity Cognition C&C ‘22 (Association for Computing Machinery, New York), 623–627.Google Scholar
• Eloundou T, Manning S, Mishkin P, Rock D (2023) GPTs are GPTs: An early look at the labor market impact potential of large language models. Working paper, Stanford University, Santa Clara, CA.Google Scholar
• Ericsson KA, Hoffman RR, Kozbelt A (2018) The Cambridge Handbook of Expertise and Expert Performance (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• Fügener A, Grahl J, Gupta A, Ketter W (2021) Will humans-in-the-loop become borgs? Merits and pitfalls of working with AI. MIS Quart. 45(3):1527–1556.Crossref, Google Scholar
• Fügener A, Grahl J, Gupta A, Ketter W (2022) Cognitive challenges in human–artificial intelligence collaboration: Investigating the path toward productive delegation. Inform. Systems Res. 33(2):678–696.Link, Google Scholar
• Gaube S, Suresh H, Raue M, Lermer E, Koch TK, Hudecek MFC, Ackery AD, et al. (2023) Non-task expert physicians benefit from correct explainable AI advice when reviewing x-rays. Sci. Rep. 13(1):1383.Crossref, Google Scholar
• Ge R, Zheng ZE, Tian X, Liao L (2021) Human–robot interaction: When investors adjust the usage of robo-advisors in peer-to-peer lending. Inform. Systems Res. 32(3):774–785.Link, Google Scholar
• Horton JJ (2023) Large language models as simulated economic agents: What can we learn from Homo silicus? Working paper, MIT Press, Cambridge, MA.Google Scholar
• Jacowitz KE, Kahneman D (1995) Measures of anchoring in estimation tasks. Personality Soc. Psych. Bull. 21(11):1161–1166.Crossref, Google Scholar
• Jentzsch S, Kersting K (2023) ChatGPT is fun, but it is not funny! Humor is still challenging large language models. Preprint, submitted June 7, https://arxiv.org/abs/2306.04563.Google Scholar
• Jussupow E, Spohrer K, Heinzl A, Gawlitza J (2021) Augmenting medical diagnosis decisions? An investigation into physicians’ decision-making process with artificial intelligence. Inform. Systems Res. 32(3):713–735.Link, Google Scholar
• Kim T, Barasz K, John LK (2019) Why am I seeing this ad? The effect of ad transparency on ad effectiveness. J. Consumer Res. 45(5):906–932.Crossref, Google Scholar
• Koch T, Denner N, Crispin M, Hohagen T (2023) Funny but not credible? Why using (many) emojis decreases message credibility and source trustworthiness. Soc. Media Soc., ePub ahead of print September 4, https://doi.org/10.1177/20563051231194584.Google Scholar
• Lee D, Hosanagar K, Nair HS (2018) Advertising content and consumer engagement on social media: Evidence from Facebook. Management Sci. 64(11):5105–5131.Link, Google Scholar
• Lysyakov M, Viswanathan S (2023) Threatened by AI: Analyzing users’ responses to the introduction of AI in a crowd-sourcing platform. Inform. Systems Res. 34(3):1191–1210.Link, Google Scholar
• Muennighoff N, Tazi N, Magne L, Reimers N (2023) MTEB: Massive text embedding benchmark. Preprint, submitted October 13, https://arxiv.org/abs/2210.07316.Google Scholar
• Naik PA, Mantrala MK, Sawyer AG (1998) Planning media schedules in the presence of dynamic advertising quality. Marketing Sci. 17(3):214–235.Link, Google Scholar
• Noy S, Zhang W (2023) Experimental evidence on the productivity effects of generative artificial intelligence. Science 381(6654):187–192.Crossref, Google Scholar
• Olson JA, Nahas J, Chmoulevitch D, Cropper SJ, Webb ME (2021) Naming unrelated words predicts creativity. Proc. Natl. Acad. Sci. USA 118(25):e2022340118.Crossref, Google Scholar
• OpenAI (2024) GPT-4 technical report. Report, OpenAI.Google Scholar
• Orwig W, Diez I, Vannini P, Beaty R, Sepulcre J (2021) Creative connections: Computational semantic distance captures individual creativity and resting-state functional connectivity. J. Cognitive Neuroscience 33(3):499–509.Crossref, Google Scholar
• Ouyang L, Wu J, Jiang X, Almeida D, Wainwright CL, Mishkin P, Zhang C, et al. (2022) Training language models to follow instructions with human feedback. Preprint, submitted March 4, https://arxiv.org/abs/2203.02155.Google Scholar
• Padmakumar V, He H (2023) Does writing with language models reduce content diversity? Preprint, submitted September 11, https://arxiv.org/abs/2309.05196.Google Scholar
• Palan S, Schitter C (2018) Prolific.ac—A subject pool for online experiments. J. Behav. Experiment. Finance 17:22–27.Crossref, Google Scholar
• Pieters R, Warlop L, Wedel M (2002) Breaking through the clutter: Benefits of advertisement originality and familiarity for brand attention and memory. Management Sci. 48(6):765–781.Link, Google Scholar
• Smith RE, Chen J, Yang X (2008) The impact of advertising creativity on the hierarchy of effects. J. Advertising 37(4):47–62.Crossref, Google Scholar
• Smith RE, MacKenzie SB, Yang X, Buchholz LM, Darley WK (2007) Modeling the determinants and effects of creativity in advertising. Marketing Sci. 26(6):819–833.Link, Google Scholar
• Stevenson C, Smal I, Baas M, Grasman R, van der Maas H (2022) Putting GPT-3′s creativity to the (alternative uses) test. Preprint, submitted June 10, https://arxiv.org/abs/2206.08932.Google Scholar
• Teixeira T, Picard R, el Kaliouby R (2014) Why, when, and how much to entertain consumers in advertisements? A web-based facial tracking field study. Marketing Sci. 33(6):809–827.Link, Google Scholar
• Tversky A, Kahneman D (1974) Judgment under uncertainty: Heuristics and biases. Science 185(4157):1124–1131.Crossref, Google Scholar
• Watson D, Clark LA, Tellegen A (1988) Development and validation of brief measures of positive and negative affect: The PANAS scales. J. Personality Soc. Psych. 54(6):1063–1070.Crossref, Google Scholar
• Zhou J (2003) When the presence of creative coworkers is related to creativity: Role of supervisor close monitoring, developmental feedback, and creative personality. J. Appl. Psych. 88(3):413–422.Crossref, Google Scholar