Digital Lyrebirds: Experimental Evidence That Voice-Based Deep Fakes Influence Trust

References

• Adam MT, Teubner T, Gimpel H (2018) No rage against the machine: How computer agents mitigate human emotional processes in electronic negotiations. Group Decision Negotiation 27(4):543–571.Crossref, Google Scholar
• Adamopoulos P, Ghose A, Todri V (2018) The impact of user personality traits on word of mouth: Text-mining social media platforms. Inform. Systems Res. 29(3):612–640.Link, Google Scholar
• Adomavicius G, Bockstedt JC, Curley SP, Zhang J (2018) Effects of online recommendations on consumers’ willingness to pay. Inform. Systems Res. 29(1):84–102.Link, Google Scholar
• Agarwal R, Kline P, Raabe J (2023) Why IVR still matters in an AI world. McKinsey Co. Insights (September 15), https://www.mckinsey.com/capabilities/operations/our-insights/operations-blog/why-ivr-still-matters-in-an-ai-world.Google Scholar
• Ahmad MI, Bernotat J, Lohan K, Eyssel F (2019) Trust and cognitive load during human-robot interaction. Preprint, submitted September 11, https://arxiv.org/abs/1909.05160.Google Scholar
• Allison TH, Warnick BJ, Davis BC, Cardon MS (2022) Can you hear me now? Engendering passion and preparedness perceptions with vocal expressions in crowdfunding pitches. J. Bus. Venturing 37(3):106193.Crossref, Google Scholar
• Almutairi Z, Elgibreen H (2022) A review of modern audio deepfake detection methods: Challenges and future directions. Algorithms 15(5):155.Crossref, Google Scholar
• Ansari A, Essegaier S, Kohli R (2000) Internet recommendation systems. J. Marketing Res. 37(3):363–375.Crossref, Google Scholar
• Aral S, Muchnik L, Sundararajan A (2009) Distinguishing influence-based contagion from homophily-driven diffusion in dynamic networks. Proc. Natl. Acad. Sci. USA 106(51):21544–21549.Crossref, Google Scholar
• Athey S, Keleher N, Spiess J (2023) Machine learning who to nudge: Causal vs. predictive targeting in a field experiment on student financial aid renewal. Preprint, submitted October 12, https://arxiv.org/abs/2310.08672.Google Scholar
• Athey S, Imbens GW, Metzger J, Munro E (2021) Using Wasserstein generative adversarial networks for the design of Monte Carlo simulations. J. Econometrics 105076.Google Scholar
• Atleson M (2023) Chatbots, deepfakes, and voice clones: AI deception for sale. Federal Trade Commission (March 20), https://www.ftc.gov/business-guidance/blog/2023/03/chatbots-deepfakes-voice-clones-ai-deception-sale.Google Scholar
• Bailenson JN, Iyengar S, Yee N, Collins NA (2008) Facial similarity between voters and candidates causes influence. Public Opinion Quart. 72(5):935–961.Crossref, Google Scholar
• Banikiotes PG, Neimeyer GJ (1981) Construct importance and rating similarity as determinants of interpersonal attraction. British J. Soc. Psych. 20:259–263.Crossref, Google Scholar
• Becker GM, Degroot MH, Marschak J (1964) Measuring utility by a single-response sequential method. Behav. Sci. 9(3):226–232.Crossref, Google Scholar
• Belin P, Boehme B, McAleer P (2017) The sound of trustworthiness: Acoustic-based modulation of perceived voice personality. PLoS One 12(10):e0185651.Crossref, Google Scholar
• Benbasat I, Wang W (2005) Trust in and adoption of online recommendation agents. J. Assoc. Inform. Systems 6(3):72–101.Google Scholar
• Bennett A, Kallus N, Schnabel T (2020) Deep generalized method of moments for instrumental variable analysis. Preprint, submitted May 29, 2019, https://arxiv.org/abs/1905.12495.Google Scholar
• Berg J, Dickhaut J, McCabe K (1995) Trust, reciprocity, and social history. Games Econom. Behav. 10(1):122–142.Crossref, Google Scholar
• Berscheid E, Hatfield E (1978) Interpersonal Attraction, 2nd ed. (Addison-Wesley Publishing Company, Reading, MA).Google Scholar
• Brunswik E (1955) Representative design and probabilistic theory in a functional psychology. Psych. Rev. 62(3):193–217.Crossref, Google Scholar
• Byrne D (1961) Interpersonal attraction and attitude similarity. J. Abnormal Soc. Psych. 62(3):713–715.Crossref, Google Scholar
• Byrne D, Clore GL (1970) A reinforcement model of evaluative responses. Personality 1(2):103–128.Google Scholar
• Camerer CF (2018) Artificial intelligence and behavioral economics. Agrawal A, Gans J, Goldfarb A, eds. The Economics of Artificial Intelligence: An Agenda (University of Chicago Press, Chicago), 587–608.Google Scholar
• Castelo N, Bos MW, Lehmann DR (2019) Task-dependent algorithm aversion. J. Marketing Res. 56(5):809–825.Crossref, Google Scholar
• Chan SW, Gunasekaran TS, Pai YS, Zhang H, Nanayakkara S (2021) Kinvoices: Using voices of friends and family in voice interfaces. Proc. ACM Human-Comput. Interaction, vol. 5 (Association for Computing Machinery, New York), 1–25.Google Scholar
• Charness G, Dufwenberg M (2006) Promises and partnership. Econometrica 74(6):1579–1601.Crossref, Google Scholar
• Charness G, Dufwenberg M (2010) Bare promises: An experiment. Econom. Lett. 107(2):281–283.Crossref, Google Scholar
• Chesney B, Citron D (2019) Deep fakes: A looming challenge for privacy, democracy, and national security. California Law Rev. 107(6):1753–1820.Google Scholar
• Choung H, David P, Ross A (2023) Trust in AI and its role in the acceptance of AI technologies. Internat. J. Human Comput. Interaction 39(9):1727–1739.Crossref, Google Scholar
• Cyr D, Head M, Larios H, Pan B (2009) Exploring human images in website design: A multi-method approach. MIS Quart. 33(3):539–566.Crossref, Google Scholar
• Dahlbäck N, Wang Q, Nass C, Alwin J (2007) Similarity is more important than expertise: Accent effects in speech interfaces. CHI ‘07: Proc. SIGCHI Conf. Human Factors Comput. Systems (Association for Computing Machinery, New York), 1553–1556.Google Scholar
• Das T, Teng BS (2004) The risk-based view of trust: A conceptual framework. J. Bus. Psych. 19:85–116.Crossref, Google Scholar
• Davis S, Mermelstein P (1980) Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Trans. Acoustics Speech Signal Processing 28(4):357–366.Crossref, Google Scholar
• Dawes RM, Faust D, Meehl PE (1989) Clinical vs. actuarial judgement. Science 243(4899):1668–1674.Crossref, Google Scholar
• DeBruine LM (2002) Facial resemblance enhances trust. Proc. Roy. Soc. London B Biol. Sci. 269(1498):1307–1312.Crossref, Google Scholar
• Dietvorst BJ, Simmons JP, Massey C (2018) Overcoming algorithm aversion: People will use imperfect algorithms if they can (even slightly) modify them. Management Sci. 64(3):1155–1170.Link, Google Scholar
• Eyssel F, Kuchenbrandt D, Bobinger S, De Ruiter L, Hegel F (2012) ‘If you sound like me, you must be more human’: On the interplay of robot and user features on human-robot acceptance and anthropomorphism. Proc. 7th Annual ACM/IEEE Internat. Conf. Human Robot Interaction (Association for Computing Machinery, New York), 125–126.Google Scholar
• Ghosh A, Fossas G (2022) Can there be art without an artist? Preprint, submitted September 16, https://arxiv.org/abs/2209.07667.Google Scholar
• Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial networks. Preprint, submitted June 10, https://arxiv.org/abs/1406.2661.Google Scholar
• Grewal R, Gupta S, Hamilton R (2021) Marketing insights from multimedia data: Text, image, audio, and video. J. Marketing Res. 58(6):1025–1033.Crossref, Google Scholar
• Groenfeldt T (2016) Citi uses voice prints to authenticate customers quickly and effortlessly. Forbes (June 27), https://www.forbes.com/sites/tomgroenfeldt/2016/06/27/citi-uses-voice-prints-to-authenticate-customers-quickly-and-effortlessly/.Google Scholar
• Guardian News and Media (2019) “Alexa, are you invading my privacy?” – The dark side of our voice assistants. The Guardian (October 9), https://www.theguardian.com/technology/2019/oct/09/alexa-are-you-invading-my-privacy-the-dark-side-of-our-voice-assistants.Google Scholar
• Guardian News and Media (2023) Disinformation reimagined: How AI could erode democracy in the 2024 US elections. The Guardian (July 19), https://www.theguardian.com/us-news/2023/jul/19/ai-generated-disinformation-us-elections.Google Scholar
• Guerouaou N, Vaiva G, Aucouturier JJ (2022) The shallow of your smile: The ethics of expressive vocal deep-fakes. Philos. Trans. Roy. Soc. B 377(1841):20210083.Crossref, Google Scholar
• Gupta A, Kannan K, Sanyal P (2018) Economic experiments in information systems. MIS Quart. 42(2):595–606.Crossref, Google Scholar
• Hegel F, Muhl C, Wrede B, Hielscher-Fastabend M, Sagerer G (2009) Understanding social robots. 2009 2nd Internat. Conf. Adv. Comput. Human Interactions (Institute of Electrical and Electronics Engineers, Piscataway, NJ), 169–174.Google Scholar
• Holmes J, Robins D (2007) Visceral and cognitive levels of credibility judgment in an authorless environment: A factor analysis of the influence of visual design. Proc. Amer. Soc. Inform. Sci. Tech. 44(1):1–10.Crossref, Google Scholar
• Hosanagar K, Fleder D, Lee D, Buja A (2014) Will the global village fracture into tribes recommender systems and their effects on consumer fragmentation. Management Sci. 60(4):805–823.Link, Google Scholar
• Howard A, Borenstein J (2019) Trust and bias in robots: These elements of artificial intelligence present ethical challenges, which scientists are trying to solve. Amer. Sci. 107(2):86–90.Crossref, Google Scholar
• Hu C, Thomas KM, Lance CE (2008) Intentions to initiate mentoring relationships: Understanding the impact of race, proactivity, feelings of deprivation, and relationship roles. J. Soc. Psych. 148(6):727–744.Crossref, Google Scholar
• Huang J, Reiley D, Riabov N (2018) Measuring consumer sensitivity to audio advertising: A field experiment on pandora Internet radio. Preprint, submitted April 22, https://dx.doi.org/10.2139/ssrn.3166676.Google Scholar
• Jia Y, Zhang Y, Weiss R, Wang Q, Shen J, Ren F, Chen Z, et al. (2018) Transfer learning from speaker verification to multispeaker text-to-speech synthesis. Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R, eds. Advances in Neural Information Processing Systems, vol. 31 (Curran Associates, Inc, Red Hook, NY).Google Scholar
• Jiang Z, Wang W, Tan BC, Yu J (2016) The determinants and impacts of aesthetics in users’ first interaction with websites. J. Management Inform. Systems 33(1):229–259.Crossref, Google Scholar
• Kaptein M, Castaneda D, Fernandez N, Nass C (2014) Extending the similarity-attraction effect: The effects of when-similarity in computer-mediated communication. J. Comput. Mediated Comm. 19(3):342–357.Crossref, Google Scholar
• Karelaia N, Hogarth RM (2008) Determinants of linear judgment: A meta-analysis of lens model studies. Psych. Bul. 134(3):404–426.Crossref, Google Scholar
• Kiesler S, Sproull L, Waters K (1996) A prisoners dilemma experiment on cooperation with people and human-like computers. J. Personality Soc. Psych. 70(1):47–65.Crossref, Google Scholar
• Kingma DP, Welling M (2019) An Introduction to Variational Autoencoders, Foundations and Trends in Machine Learning, vol. 12 (now Publishers, Hanover, MA), 307–392.Crossref, Google Scholar
• Kirkby A, Baumgarth C, Henseler J (2023) To disclose or not disclose, is no longer the question–effect of AI-disclosed brand voice on brand authenticity and attitude. J. Product Brand Management 32(7):1108–1122.Crossref, Google Scholar
• Kleinberg J, Lakkaraju H, Leskovec J, Ludwig J, Mullainathan S (2017) Human decisions and machine predictions. Quart. J. Econom. 133(1):237–293.Crossref, Google Scholar
• Lee B (2006) Empathy, androids and ‘authentic experience.’ Connection Sci. 18(4):419–428.Crossref, Google Scholar
• Lee MK (2018) Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management. Big Data Soc. 5(1):2053951718756684.Crossref, Google Scholar
• Lee JD, See KA (2004) Trust in automation: Designing for appropriate reliance. Human Factors 46(1):50–80.Crossref, Google Scholar
• Lee MK, Kiesler S, Forlizzi J (2011) Mining behavioral economics to design persuasive technology for healthy choices. CHI ‘11 Proc. SIGCHI Conf. Human Factors Comput. Systems (Association for Computing Machinery, New York), 325–334.Google Scholar
• Lewis M, Sycara K, Walker P (2018) The role of trust in human-robot interaction. Abbass H, Scholz J, Reid D, eds. Foundations of Trusted Autonomy (Springer, Cham, Switzerland), 135–159.Crossref, Google Scholar
• Li P, Tuzhilin A (2019) Toward controllable and personalized review generation. Preprint, submitted September 30, https://arxiv.org/abs/1910.03506.Google Scholar
• Libai B, Bart Y, Gensler S, Hofacker CF, Kaplan A, Kötterheinrich K, Kroll EB (2020) Brave new world? On AI and the management of customer relationships. J. Marketing 51:44–56.Google Scholar
• Liu BQ, Goodhue DL (2012) Two worlds of trust for potential e-commerce users: Humans as cognitive misers. Inform. Systems Res. 23(4):1246–1262.Link, Google Scholar
• Logg JM, Minson JA, Moore DA (2019) Algorithm appreciation: People prefer algorithmic to human judgment. Organ. Behav. Human Decision Processes 151:90–103.Crossref, Google Scholar
• Longoni C, Fradkin A, Cian L, Pennycook G (2022) News from generative artificial intelligence is believed less. 2022 ACM Conf. Fairness Accountability Transparency (Association for Computing Machinery, New York), 97–106.Google Scholar
• Luo X, Qin MS, Fang Z, Qu Z (2021) Artificial intelligence coaches for sales agents: Caveats and solutions. J. Marketing 85(2):14–32.Crossref, Google Scholar
• Luo X, Tong S, Fang Z, Qu Z (2019) Frontiers: Machines vs. humans: The impact of artificial intelligence chatbot disclosure on customer purchases. Marketing Sci. 38(6):937–947.Abstract, Google Scholar
• Mahmud H, Islam AN, Ahmed SI, Smolander K (2022) What influences algorithmic decision-making? A systematic literature review on algorithm aversion. Tech. Forecasting Soc. Change 175:121390.Crossref, Google Scholar
• March C (2019) The behavioral economics of artificial intelligence: Lessons from experiments with computer players. CESifo Working Paper No. 7926, CESifo, Munich.Google Scholar
• McAleer P, Todorov A, Belin P (2014) How do you say ‘hello’? Personality impressions from brief novel voices. PLoS One 9(3):e90779.Crossref, Google Scholar
• McCabe K, Houser D, Ryan L, Smith V, Trouard T (2001) A functional imaging study of cooperation in two-person reciprocal exchange. Proc. Natl. Acad. Sci. USA 98(20):11832–11835.Crossref, Google Scholar
• McGinn C, Torre I (2019) Can you tell the robot by the voice? An exploratory study on the role of voice in the perception of robots. 2019 14th ACM/IEEE Internat. Conf. Human Robot Interaction (HRI) (Institute of Electrical and Electronics Engineers, Piscataway, NJ), 211–221.Google Scholar
• Mcuba M, Singh A, Ikuesan RA, Venter H (2023) The effect of deep learning methods on deepfake audio detection for digital investigation. Procedia Comput. Sci. 219:211–219.Crossref, Google Scholar
• Mori M (1970) The uncanny valley. Energy 7:33–35.Google Scholar
• Morozovaite V (2023) The future of anticompetitive self-preferencing: Analysis of hypernudging by voice assistants under article 102 TFEU. Eur. Competition J. 19(3):410–448.Crossref, Google Scholar
• Mozafari N, Weiger WH, Hammerschmidt M (2022) Trust me, I’m a bot–repercussions of chatbot disclosure in different service frontline settings. J. Service Management 33(2):221–245.Crossref, Google Scholar
• Möhlmann M, Zalmanson L, Henfridsson O, Gregory R (2021) Algorithmic management of work on online labor platforms: When matching meets control. Management Inform. Systems Quart. 45(4):1999–2022.Crossref, Google Scholar
• Nass C, Lee KM (2001) Does computer-synthesized speech manifest personality? Experimental tests of recognition, similarity-attraction, and consistency-attraction. J. Experiment. Psych. Appl. 7(3):171–181.Crossref, Google Scholar
• Nass C, Moon Y (2000) Machines and mindlessness: Social responses to computers. J. Soc. Issues 56(1):81–103.Crossref, Google Scholar
• Nass C, Moon Y, Green N (1997) Are machines gender neutral? Gender-stereotypic responses to computers with voices. J. Appl. Soc. Psych. 27(10):864–876.Crossref, Google Scholar
• Ozogul G, Johnson AM, Atkinson RK, Reisslein M (2013) Investigating the impact of pedagogical agent gender matching and learner choice on learning outcomes and perceptions. Comput. Ed. 67:36–50.Crossref, Google Scholar
• Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans. Knowledge Data Engrg. 22(10):1345–1359.Crossref, Google Scholar
• Park KF, Shapira Z (2017) Risk and uncertainty. Augier M, Teece DJ, eds. The Palgrave Encyclopedia of Strategic Management (Palgrave Macmillan, London), 1479–1485.Crossref, Google Scholar
• Pörschmann C (2000) Influences of bone conduction and air conduction on the sound of one’s own voice. Acta Acustica united with Acustica 86(6):1038–1045.Google Scholar
• Promberger M, Baron J (2006) Do patients trust computers? J. Behav. Decision Making 19(5):455–468.Crossref, Google Scholar
• Rousey C, Holzman PS (1967) Recognition of one’s own voice. J. Personality Soc. Psych. 6(4):464–466.Crossref, Google Scholar
• Samat S, Péer E, Acquisti A (2018) Can digital face-morphs influence attitudes and online behaviors? 14th Sympos. Usable Privacy Security (SOUPS 2018) (USENIX Association, Baltimore), 117–125.Google Scholar
• Sanfey AG, Rilling JK, Aronson JA, Nystrom LE, Cohen JD (2003) The neural basis of economic decision-making in the ultimatum game. Science 300(5626):1755–1758.Crossref, Google Scholar
• Schanke S, Burtch G, Ray G (2021) Estimating the impact of “humanizing” customer service chatbots. Inform. Systems Res. 32(3):736–751.Link, Google Scholar
• Singh R, Tay YY, Sankaran K (2017) Causal role of trust in interpersonal attraction from attitude similarity. J. Soc. Personal Relationships 34(5):717–731.Crossref, Google Scholar
• Smith VL, Suchanek GL, Williams AW (1988) Bubbles, crashes, and endogenous expectations in experimental spot asset markets. Econometrica 56(5):1119–1151.Crossref, Google Scholar
• Tambe P, Cappelli P, Yakubovich V (2019) Artificial intelligence in human resources management: Challenges and a path forward. California Management Rev. 61(4):15–42.Crossref, Google Scholar
• Thaler RH (2018) Behavioral economics: Past, present, and future. Revista de Economia Institucional 20(38):9–43.Crossref, Google Scholar
• Titze IR (1989) Physiologic and acoustic differences between male and female voices. J. Acoustical Soc. Amer. 85(4):1699–1707.Crossref, Google Scholar
• Todorov A, Pakrashi M, Oosterhof NN (2009) Evaluating faces on trustworthiness after minimal time exposure. Soc. Cognition 27(6):813–833.Crossref, Google Scholar
• Tong S, Jia N, Luo X, Fang Z (2021) The Janus face of artificial intelligence feedback: Deployment vs. disclosure effects on employee performance. Strategic Management J. 42(9):1600–1631.Crossref, Google Scholar
• Torta E, Dijk EV, Ruijten PAM, Cuijpers RH (2013) The ultimatum game as measurement tool for anthropomorphism in human–robot interaction. Herrmann G, Pearson MJ, Lenz A, Bremner P, Spiers A, Leonards U, eds. Soc. Robotics, Lecture Notes in Computer Science, vol. 8239 (Springer, Cham, Switzerland), 209–217.Google Scholar
• Tractinsky N (2015) Aesthetics in information technology: Motivation and future research directions. Zhang P, Galletta DF, eds. Human-Computer Interaction and Management Information Systems: Foundations, 1st ed. (Routledge, New York), 344–361.Google Scholar
• Trainer T, Taylor JR, Stanton CJ (2020) Choosing the best robot for the job: Affinity bias in human-robot interaction. Wagner AR, eds. Soc. Robotics ICSR 2020, Lecture Notes in Computer Science, vol. 12483 (Springer, Cham, Switzerland), 490–501.Google Scholar
• Turow J (2021) The Voice Catchers: How Marketers Listen in to Exploit Your Feelings, Your Privacy, and Your Wallet (Yale University Press, New Haven, CT).Google Scholar
• Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Proc. 31st Internat. Conf. Neural Inform. Processing Systems (NIPS'17) (Curran Associates Inc., Red Hook, NY), 6000–6010.Google Scholar
• Walther JB, Tidwell LC (1995) Nonverbal cues in computer-mediated communication, and the effect of chronemics on relational communication. J. Organ. Comput. 5(4):355–378.Crossref, Google Scholar
• Yeomans M, Shah A, Mullainathan S, Kleinberg J (2019) Making sense of recommendations. J. Behav. Decision Making 32(4):403–414.Crossref, Google Scholar
• Yeong Tan DT, Singh R (1995) Attitudes and attraction: A developmental study of the similarity-attraction and dissimilarity-repulsion hypotheses. Personality Soc. Psych. Bull. 21(9):975–986.Crossref, Google Scholar
• Yeung K (2019) ‘Hypernudge’: Big data as a mode of regulation by design. Inform. Comm. Soc. 20(1):118–136.Crossref, Google Scholar
• Young A (2019) Channeling Fisher: Randomization tests and the statistical insignificance of seemingly significant experimental results. Quart. J. Econom. 134(2):557–598.Crossref, Google Scholar
• Zhou E, Lee D (2024) Generative artificial intelligence, human creativity, and art. PNAS Nexus 3(3):pgae052.Crossref, Google Scholar