The Language That Drives Engagement: A Systematic Large-scale Analysis of Headline Experiments

References

• Anderson R (2020) Social Media Marketing 2020: How to Crush It with Instagram, Facebook, YouTube & Co.: Proven Strategies for Beginners & Advanced Marketers to Build Your Personal Brand, Get Followers and Make Money Online (Independently Published).Google Scholar
• Andersen PA, Blackburn TR (2004) An experimental study of language intensity and response rate in email surveys. Communication Rep. 17(2):73–84.Google Scholar
• Anderson ET, Simester DI (2014) Reviews without a purchase: Low ratings, loyal customers, and deception. J. Marketing Res. 51(3):249–269.Crossref, Google Scholar
• Banerjee A, Urminsky O (2022) What you are getting and what you will be getting: Testing whether verb tense affects intertemporal choices. J. Experimental Psych. General 151(10):2342.Google Scholar
• Banerjee A, Urminsky O (2024) Does big-data correlational analysis predict causal effects of language on decisions? Retrieved October 8, 2024, https://osf.io/v3zu5/.Google Scholar
• Bartels DM, Hastie R, Urminsky O (2018) Connecting laboratory and field research in judgment and decision making: Causality and the breadth of external validity. J. Appl. Res. Memory Cognition 7(1):11–15.Google Scholar
• Berger J, Milkman KL (2012) What makes online content viral? J. Marketing Res. 49(2):192–205.Crossref, Google Scholar
• Berger J, Sherman G, Ungar L (2020) TextAnalyzer. http://textanalyzer.org/about.Google Scholar
• Boyd RL, Ashokkumar A, Seraj S, Pennebaker JW (2022) The development and psychometric properties of LIWC-22. Report, University of Texas at Austin, Austin.Google Scholar
• Brady WJ, Van Bavel JJ (2021) Estimating the effect size of moral contagion in online networks: A pre-registered replication and meta-analysis. Preprint, submitted April 7, https://doi.org/10.31219/osf.io/s4w2x.Google Scholar
• Büschken J, Allenby GM (2016) Sentence-based text analysis for customer reviews. Marketing Sci. 35(6):953–975.Link, Google Scholar
• Chakraborty A, Paranjape B, Kakarla S, Ganguly N (2016) Stop clickbait: Detecting and preventing clickbaits in online news media. 2016 IEEE/ACM Internat. Conf. Adv. Soc. Networks Analysis Mining (ASONAM) (IEEE Press, Piscataway, NJ), 9–16.Google Scholar
• Chambers CD, Dienes Z, McIntosh RD, Rotshtein P, Willmes K (2015) Registered reports: Realigning incentives in scientific publishing. Cortex 66:A1–A2.Crossref, Google Scholar
• Choi VK, Shrestha S, Pan X, Gelfand MJ (2022) When danger strikes: A linguistic tool for tracking America’s collective response to threats. Proc. Natl. Acad. Sci. USA 119(4):e2113891119.Crossref, Google Scholar
• Christian (2012) The A/B Test: Inside the technology that’s changing the rules of business. WIRED (April 25), https://www.wired.com/2012/04/ff-abtesting/.Google Scholar
• Dor D (2003) On newspaper headlines as relevance optimizers. J. Pragmatics 35(5):695–721.Crossref, Google Scholar
• Eckles D (2024) Pervasive randomization problems, here with headline experiments. Statistical Modeling, Causal Inference, and Social Science (June 20), https://statmodeling.stat.columbia.edu/2024/06/20/pervasive-randomization-problems-here-with-headline-experiments/.Google Scholar
• Eichstaedt JC, Smith RJ, Merchant RM, Ungar LH, Crutchley P, Preoţiuc-Pietro D, Asch DA, Schwartz HA (2018) Facebook language predicts depression in medical records. Proc. Natl. Acad. Sci. USA 115(44):11203–11208.Crossref, Google Scholar
• Elder RS, Krishna A (2010) The effects of advertising copy on sensory thoughts and perceived taste. J. Consumer Res. 36(5):748–756.Google Scholar
• Frampton B (2015) Clickbait: The changing face of online journalism. BBC News (September 14), https://www.bbc.com/news/uk-wales-34213693.Google Scholar
• Formanowicz MM, Cisłak A, Horvath LK, Sczesny S (2015) Capturing socially motivated linguistic change: How the use of gender-fair language affects support for social initiatives in Austria and Poland. Front. Psychol. 6:147750.Crossref, Google Scholar
• Goswami I, Urminsky O (2020) No substitute for the real thing: The importance of in-context field experiments in fundraising. Marketing Sci. 39(6):1052–1070.Link, Google Scholar
• Guadagno RE, Cialdini RB (2005) Online persuasion and compliance: Social influence on the Internet and beyond. The Social Net: The Social Psychology of the Internet (Oxford University Press, New York), 91–113.Google Scholar
• Hansen J, Wänke M (2010) Truth from language and truth from fit: The impact of linguistic concreteness and level of construal on subjective truth. Pers. Soc. Psychol. Bull. 36(11):1576–1588.Crossref, Google Scholar
• Heimbach I, Schiller B, Strufe T, Hinz O (2015) Content virality on online social networks: Empirical evidence from Twitter, Facebook, and Google+ on German news websites. Proc. 26th ACM Conf. Hypertext Soc. Media (ACM, New York), 39–47.Google Scholar
• Heller L (2018) This Stanford scholar learned why clickbait will surprise you. Stanford News (March 21), https://news.stanford.edu/stories/2018/03/this-stanford-scholar-learned-clickbait-will-surprise.Google Scholar
• Hsu T (2019) You will be shocked by this article. New York Times (October 4), https://www.nytimes.com/2019/10/04/business/media/online-advertising-chumbox-merger.html.Google Scholar
• Karabell (2017) Upworthy’s Quest to Engineer Optimism for an Anxious Age. WIRED (May 1), https://www.wired.com/2017/05/upworthys-quest-engineer-optimism-anxious-age/.Google Scholar
• Kim T, Kim J (2021) How spatial distance and message strategy in cause-related marketing ads influence consumers’ ad believability and attitudes. Sustainability 13(12):6775.Crossref, Google Scholar
• Kim JH, Mantrach A, Jaimes A, Oh A (2016) How to compete online for news audience: Modeling words that attract clicks. Proc. 22nd ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (ACM, New York), 1645–1654.Google Scholar
• Kincaid JP, Fishburne RP Jr, Rogers RL, Chissom BS (1975) Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel (Institute for Simulation and Training, University of Central Florida, Orlando).Crossref, Google Scholar
• Kuiken J, Schuth A, Spitters M, Marx M (2017) Effective headlines of newspaper articles in a digital environment. Digital Journalism 5(10):1300–1314.Crossref, Google Scholar
• Labroo AA, Lambotte S, Zhang Y (2009) The “name-ease” effect and its dual impact on importance judgments. Psychol. Sci. 20(12):1516–1522.Crossref, Google Scholar
• Lai L, Farbrot A (2014) What makes you click? The effect of question headlines on readership in computer-mediated communication. Social Influence 9(4):289–299.Google Scholar
• Lambert AJ, Eadeh FR, Hanson EJ (2019) Anger and its consequences for judgment and behavior: Recent developments in social and political psychology. Adv. Exp. Soc. Psychol. 59:103–173.Crossref, Google Scholar
• Larrimore L, Jiang L, Larrimore J, Markowitz D, Gorski S (2011) Peer to peer lending: The relationship between language features, trustworthiness, and persuasion success. J. Appl. Commun. Res. 39(1):19–37.Crossref, Google Scholar
• Liberman N, Trope Y, Wakslak C (2007) Construal level theory and consumer behavior. J. Consum. Psychol. 17(2):113–117.Crossref, Google Scholar
• Loewenstein G (1994) The psychology of curiosity: A review and reinterpretation. Psychol. Bull. 116(1):75.Crossref, Google Scholar
• Loewenstein G, Lerner JS (2003) The role of affect in decision making. Handbook of Affective Sciences (Oxford University Press, Oxford, UK).Google Scholar
• Mahler J (2010) The king of content. Columbia J. Rev. (May 4), https://www.cjr.org/feature/the_king_of_content.php.Google Scholar
• Matheson K, Zanna MP (1989) Persuasion as a function of self-awareness in computer-mediated communication. Social Behav. 4(2):99–111.Google Scholar
• Matias JN, Munger K, Le Quere MA, Ebersole C (2021) The Upworthy Research Archive, a time series of 32,487 experiments in US media. Sci. Data 8(1):195.Crossref, Google Scholar
• Menon S, Soman D (2002) Managing the power of curiosity for effective web advertising strategies. J. Advert. 31(3):1–14.Crossref, Google Scholar
• Miller R, Charles EYA (2016) A psychological based analysis of marketing email subject lines. 2016 Sixteenth Internat. Conf. Adv. ICT Emerging Regions (ICTer) (IEEE, Piscataway, NJ), 58–65.Google Scholar
• Moore DJ, Lee SP (2012) How advertising influences consumption impulses. J. Advert. 41(3):107–120.Crossref, Google Scholar
• Morand DA (1995) The role of behavioral formality and informality in the enactment of bureaucratic vs. organic organizations. Acad. Management Rev. 20(4):831–872.Crossref, Google Scholar
• Munger K, Luca M, Nagler J, Tucker J (2018) The effect of clickbait in online media. Retrieved October 12, 2024, http://kmunger.github.io/pdfs/clickbait.pdf.Google Scholar
• Nelson MR, Hitchon JC (1999) Loud tastes, colored fragrances, and scented sounds: How and when to mix the senses in persuasive communications. J. Mass Commun. Quart. 76(2):354–372.Google Scholar
• Nenkov GY, Scott ML (2014) “So cute I could eat it up”: Priming effects of cute products on indulgent consumption. J. Consum. Res. 41(2):326–341.Crossref, Google Scholar
• Netzer O, Lemaire A, Herzenstein M (2019) When words sweat: Identifying signals for loan default in the text of loan applications. J. Marketing Res. 56(6):960–980.Crossref, Google Scholar
• Oatley K (1999) Meetings of minds: Dialogue, sympathy, and identification, in reading fiction. Poetics 26(5–6):439–454.Crossref, Google Scholar
• O’Donovan (2013) “If you’re not feeling it, don’t write it”: Upworthy’s social success depends on gut-checking “regular people.” Nieman Laboratory (June 20), https://www.niemanlab.org/2013/06/if-youre-not-feeling-it-dont-write-it-upworthys-social-success-depends-on-gut-checking-regular-people/.Google Scholar
• Oppenheimer DM (2006) Consequences of erudite vernacular utilized irrespective of necessity: Problems with using long words needlessly. Appl. Cogn. Psychol. 20(2):139–156.Crossref, Google Scholar
• Packard G, Berger J (2020) Thinking of you: How second-person pronouns shape cultural success. Psychol. Sci. 31(4):397–407.Crossref, Google Scholar
• Pengnate SF (2019) Shocking secret you won’t believe! Emotional arousal in clickbait headlines: An eye-tracking analysis. Online Inform. Rev. 43(7):1136–1150.Google Scholar
• Pennebaker JW, Boyd RL, Jordan K, Blackburn K (2015) The Development and Psychometric Properties of LIWC2015 (University of Texas at Austin, Austin).Google Scholar
• Pennebaker JW, King LA (1999) Linguistic styles: Language use as an individual difference. J. Personality Soc. Psychol. 77(6):1296.Crossref, Google Scholar
• Reimers N, Gurevych I (2019) Sentence-bert: Sentence embeddings using siamese bert-networks. Preprint, submitted August 27, https://arxiv.org/abs/1908.10084.Google Scholar
• Rennekamp K (2012) Processing fluency and investors’ reactions to disclosure readability. J. Account. Res. 50(5):1319–1354.Crossref, Google Scholar
• Richardson M, Dominowska E, Ragno R (2007) Predicting clicks: Estimating the click-through rate for new ads. Proc. 16th Internat. Conf. World Wide Web (ACM, New York), 521–530.Google Scholar
• Roche JM, Peters B, Dale R (2015) “Your tone says it all”: The processing and interpretation of affective language. Speech Commun. 66:47–64.Crossref, Google Scholar
• Rosenkranz S, Vringer K, Dirkmaat T, van den Broek E, Abeelen C, Travaille A (2017) Using behavioral insights to make firms more energy efficient: A field experiment on the effects of improved communication. Energy Policy 108:184–193.Google Scholar
• Ruan B, Hsee CK, Lu ZY (2018) The teasing effect: An underappreciated benefit of creating and resolving an uncertainty. J. Marketing Res. 55(4):556–570.Crossref, Google Scholar
• Sadoski M, Goetz E, Rodriguez M (2000) Engaging texts: Effects of concreteness on comprehensibility, interest, and recall in four text types. J. Educ. Psychol. 92:85–95.Crossref, Google Scholar
• Sallis A, Harper H, Sanders M (2018) Effect of persuasive messages on National Health Service Organ Donor Registrations: A pragmatic quasi-randomised controlled trial with one million UK road taxpayers. Trials 19(1):1–10.Google Scholar
• Sanders (2017) Upworthy was one of the hottest sites ever. You won’t believe what happened next. NPR.org (June 20), https://www.npr.org/sections/alltechconsidered/2017/06/20/533529538/upworthy-was-one-of-the-hottest-sites-ever-you-wont-believe-what-happened-next.Google Scholar
• Scacco JM, Muddiman A (2020) The curiosity effect: Information seeking in the contemporary news environment. New Media Soc. 22(3):429–448.Crossref, Google Scholar
• She J, Zhang T (2019) The impact of headline features on the attraction of online financial articles. Internat. J. Web Inform. Systems 15(5):510–534.Google Scholar
• Smith JR, Louis WR, Abraham C (2018) When and how does normative feedback reduce intentions to drink irresponsibly? An experimental investigation. Addict. Res. Theory 26(4):256–266.Crossref, Google Scholar
• Stein (2019) The rise of clickbait brands. Advertising Week 360 • AW360, https://www.advertisingweek360.com/the-rise-of-clickbait-brands/.Google Scholar
• Suciu P (2020) From scams to mainstream headlines, clickbait is on the rise. Forbes (February 10), https://www.forbes.com/sites/petersuciu/2020/02/10/clickbait-how-baby-yoda-is-mourning-kobe-bryant-and-kirk-douglas-while-giving-up-vaping-and-cbd-to-fight-coronavirus/.Google Scholar
• Tausczik YR, Pennebaker JW (2010) The psychological meaning of words: LIWC and computerized text analysis methods. J. Lang. Soc. Psychol. 29(1):24–54.Crossref, Google Scholar
• Tenenboim O, Cohen AA (2015) What prompts users to click and comment: A longitudinal study of online news. Journalism 16(2):198–217.Crossref, Google Scholar
• Tirunillai S, Tellis GJ (2014) Mining marketing meaning from online chatter: Strategic brand analysis of big data using latent Dirichlet allocation. J. Marketing Res. 51(4):463–479.Crossref, Google Scholar
• Tsai CI, Thomas M (2011) When does feeling of fluency matter? How abstract and concrete thinking influence fluency effects. Psych. Sci. 22(3):348–354.Google Scholar
• Urminsky O, Hansen C, Chernozhukov V (2016) Using double-lasso regression for principled variable selection. Preprint, submitted February 18, http://dx.doi.org/10.2139/ssrn.2733374.Google Scholar
• Valenzuela S, Piña M, Ramírez J (2017) Behavioral effects of framing on social media users: How conflict, economic, human interest, and morality frames drive news sharing. J. Commun. 67(5):803–826.Crossref, Google Scholar
• Wagner TF, Baccarella CV, Voigt K-I (2017) Framing social media communication: Investigating the effects of brand post appeals on user interaction. Eur. Management J. 35(5):606–616.Crossref, Google Scholar
• Wei W, Wan X (2017). Learning to identify ambiguous and misleading news headlines. Preprint, submitted May 17, https://arxiv.org/abs/1705.06031.Google Scholar
• Wood W (2000) Attitude change: Persuasion and social influence. Annu. Rev. Psychol. 51(1):539–570.Crossref, Google Scholar