Engaging Customers with AI in Online Chats: Evidence from a Randomized Field Experiment

References

• Adam M, Wessel M, Benlian A (2020) AI-based chatbots in customer service and their effects on user compliance. Electronic Markets 31(2):427–445.Crossref, Google Scholar
• Agrawal A, Gans JS, Goldfarb A (2019) Exploring the impact of artificial intelligence: Prediction versus judgment. Inform. Econom. Policy 47:1–6.Crossref, Google Scholar
• Agrawal A, Gans JS, Goldfarb A (2022) Power and Prediction: The Disruptive Economics of Artificial Intelligence (Harvard Business Review Press, Brighton, MA).Google Scholar
• Allen RT, Choudhury P (2021) Algorithm-augmented work and domain experience: The countervailing forces of ability and aversion. Organ. Sci. 33(1):149–169.Link, Google Scholar
• Arora N, Chakraborty I, Nishimura Y (2024) AI-human hybrids for marketing research: Leveraging LLMs as collaborators. J. Marketing 89(2):43–70.Google Scholar
• Athey S, Imbens GW (2022) Design-based analysis in difference-in-differences settings with staggered adoption. J. Econometrics 226:62–79.Crossref, Google Scholar
• Belanche D, Casaló LV, Flavián C, Schepers J (2020) Robots or frontline employees? Exploring customers’ attributions of responsibility and stability after service failure or success. J. Service Management 31(2):267–289.Crossref, Google Scholar
• Borusyak K, Jaravel X, Spiess J (2024) Revisiting event-study designs: Robust and efficient estimation. Rev. Econom. Stud. 91(6):3253–3285.Google Scholar
• Bowman D, Narayandas D (2001) Managing customer-initiated contacts with manufacturers: The impact on share of category requirements and word-of-mouth behavior. J. Marketing Res. 38(3):281–297.Crossref, Google Scholar
• Bowman D, Narayandas D (2004) Linking customer management effort to customer profitability in business markets. J. Marketing Res. 41(4):433–447.Crossref, Google Scholar
• Brynjolfsson E, Li D, Raymond L (2023) Generative AI at work. NBER Working Paper No. 31161, National Bureau of Economic Research, Cambridge, MA.Google Scholar
• Callaway B, Sant’Anna PHC (2021) Difference-in-differences with multiple time periods. J. Econometrics 225:200–230.Crossref, Google Scholar
• Castelo N, Katona Z, Li P, Sarvary M (2024) How AI outperforms humans at creative idea generation. Preprint, submitted April 5, https://doi.org/10.2139/ssrn.4751779.Google Scholar
• Chakraborty I, Chiong L, Dover H, Sudhir K (2024) AI and AI-human based salesforce hiring using interview videos. Marketing Sci. 44(1):30–53.Link, Google Scholar
• Chevalier JA, Mayzlin D (2006) The effect of word of mouth on sales: Online book reviews. J. Marketing Res. 43(3):345–354.Crossref, Google Scholar
• Chiu A, Lan X, Liu Z, Xu Y (2023) Causal panel analysis under parallel trends: Lessons from a large reanalysis study*. Preprint, submitted July 5, https://doi.org/10.2139/ssrn.4490035.Google Scholar
• Choudhury P, Starr E, Agarwal R (2020) Machine learning and human capital complementarities: Experimental evidence on bias mitigation. Strategic Management J. 41:1381–1411.Crossref, Google Scholar
• Crolic C, Thomaz F, Hadi R, Stephen AT (2022) Blame the bot: Anthropomorphism and anger in customer–Chatbot interactions. J. Marketing 86(1):132–148.Crossref, Google Scholar
• Cunningham S (2021) Causal Inference: The Mixtape (Yale University Press, New Haven, CT).Crossref, Google Scholar
• de Chaisemartin C, D’Haultfoeuille X (2020) Two-way fixed effects estimators with heterogeneous treatment effects. Amer. Econom. Rev. 110(9):2964–2996.Google Scholar
• Dell’Acqua F, McFowland E, Mollick ER, Lifshitz-Assaf H, Kellogg K, Rajendran S, Krayer L, et al. (2023) Navigating the jagged technological frontier: Field experimental evidence of the effects of AI on knowledge worker productivity and quality. Harvard Business School Working Paper No. 24-013, Harvard Business School, Boston.Google Scholar
• Dietvorst BJ, Simmons JP, Massey C (2015) Algorithm aversion: People erroneously avoid algorithms after seeing them err. J. Experiment. Psych. General 144(1):114–126.Crossref, Google Scholar
• Dubey A, Jauhri A, Pandey A, Kadian A, Al-Dahle A, Letman A, et al. (2024) The Llama 3 herd of models. Preprint, submitted July 31, https://arxiv.org/abs/2407.21783v2.Google Scholar
• Fu R, Huang Y, Singh PV (2021) Crowds, lending, machine, and bias. Inform. Systems Res. 32:72–92.Link, Google Scholar
• Goodman-Bacon A (2021) Difference-in-differences with variation in treatment timing. J. Econometrics 225:254–277.Crossref, Google Scholar
• Heskett JL, Jones TO, Loveman GW, Sasser WE Jr, Schlesinger LA (1994) Putting the service-profit chain to work. Harvard Bus. Rev. 72(2):164–174.Google Scholar
• Karlinsky-Shichor Y, Netzer O (2023) Automating the B2B salesperson pricing decisions: A human-machine hybrid approach. Marketing Sci. 43(1):138–157.Google Scholar
• Khandker SR, Koolwal GB, Samad HA (2010) Handbook on Impact Evaluation: Quantitative Methods and Practices (World Bank, Washington, DC).Google Scholar
• Leung E, Paolacci G, Puntoni S (2018) Man versus machine: Resisting automation in identity-based consumer behavior. J. Marketing Res. 55(6):818–831.Crossref, Google Scholar
• Liu Y (2006) Word of mouth for movies: Its dynamics and impact on box office revenue. J. Marketing 70(3):74–89.Crossref, Google Scholar
• Liu L, Wang Y, Xu Y (2024) A practical guide to counterfactual estimators for causal inference with time-series cross-sectional data. Amer. J. Political Sci. 68:160–176.Crossref, Google Scholar
• Luo X, Qin MS, Fang Z, Qu Z (2021) Artificial intelligence coaches for sales agents: Caveats and solutions. J. Marketing 85(2):14–32.Crossref, Google Scholar
• Luo X, Tong S, Fang Z, Qu Z (2019) Frontiers: Machines vs. humans: The impact of artificial intelligence chatbot disclosure on customer purchases. Marketing Sci. 38(6):937–947.Abstract, Google Scholar
• Mende M, Scott ML, van Doorn J, Grewal D, Shanks I (2019) Service robots rising: How humanoid robots influence service experiences and elicit compensatory consumer responses. J. Marketing Res. 56(4):535–556.Crossref, Google Scholar
• Obermeyer Z, Powers B, Vogeli C, Mullainathan S (2019) Dissecting racial bias in an algorithm used to manage the health of populations. Sci. 366(6464):447–453.Google Scholar
• Qiu L, Singh PV, Srinivasan K (2023) Consumer risk preferences elicitation from large language models. Preprint, submitted July 31, https://doi.org/10.2139/ssrn.4526072.Google Scholar
• Rake R, Jangra H, Kumar V (2021) Conversational AI market outlook—2030. Accessed June 2022, https://www.alliedmarketresearch.com/conversational-ai-market-A13682.Google Scholar
• Seamans R, Zhu F (2014) Responses to entry in multi-sided markets: The impact of Craigslist on local newspapers. Management Sci. 60(2):476–493.Link, Google Scholar
• Sun J, Zhang DJ, Hu H, Van Mieghem JA (2021) Predicting human discretion to adjust algorithmic prescription: A large-scale field experiment in warehouse operations. Management Sci. 68(2):846–865.Link, Google Scholar
• Wang K, Goldfarb A (2017) Can offline stores drive online sales? J. Management Rev. 54(5):706–719.Google Scholar
• Wooldridge J (2021) Two-way fixed effects, the two-way Mundlak regression, and difference-in-differences estimators. Preprint, submitted August 18, https://doi.org/10.2139/ssrn.3906345.Google Scholar
• Xu Y (2024) Causal inference with time-series cross-sectional data: A reflection. Box-Steffensmeier JM, Christenson DP, Sinclair-Chapman V, eds. Oxford Handbook of Engaged Methodological Pluralism in Political Science, vol. 1 (Oxford Academic, Oxford).Crossref, Google Scholar
• Yang J, Eckles D, Dhillon P, Aral S (2023) Targeting for long-term outcomes. Management Sci. 70(6):3841–3855.Link, Google Scholar
• Zhang W, Liu Z, Liu X, Muller E (2023) Doubling revenues by adopting livestream shopping: A synthetic DiD approach. Preprint, submitted January 6, https://doi.org/10.2139/ssrn.4318978.Google Scholar
• Zhang S, Mehta N, Singh P, Srinivasan K (2021) Can an AI algorithm mitigate racial economic inequality? An analysis in the context of Airbnb. Marketing Sci. 40(5):813–820.Link, Google Scholar