Optimizing User Engagement Through Adaptive Ad Sequencing

References

• Aguirregabiria V, Mira P (2002) Swapping the nested fixed point algorithm: A class of estimators for discrete Markov decision models. Econometrica 70(4):1519–1543.Crossref, Google Scholar
• Ansari A, Mela CF (2003) E-customization. J. Marketing Res. 40(2):131–145.Crossref, Google Scholar
• Aravindakshan A, Naik PA (2011) How does awareness evolve when advertising stops? The role of memory. Marketing Lett. 22(3):315–326.Crossref, Google Scholar
• Arnosti N, Beck M, Milgrom P (2016) Adverse selection and auction design for Internet display advertising. Amer. Econom. Rev. 106(10):2852–2866.Crossref, Google Scholar
• Bellman R (1966) Dynamic programming. Science 153(3731):34–37.Crossref, Google Scholar
• Bellman R, Dreyfus S (1959) Functional approximations and dynamic programming. Math. Tables Other Aids Comput. 13(68):247–251.Crossref, Google Scholar
• Chen T, Guestrin C (2016) XGBoost: A scalable tree boosting system. Krishnapuram B, ed. Proc. 22nd ACM SIGKDD Internat. Conf. on Knowledge Discovery and Data Mining (ACM, New York), 785–794.Google Scholar
• Despotakis S, Ravi R, Sayedi A (2021) First-price auctions in online display advertising. J. Marketing Res. 58(5):888–907.Crossref, Google Scholar
• Dubé J-P, Hitsch GJ, Manchanda P (2005) An empirical model of advertising dynamics. Quant. Marketing Econom. 3(2):107–144.Crossref, Google Scholar
• eMarketer (2018) Mobile in-app ad spending. Accessed April 19, 2018, https://forecasts-na1.emarketer.com/584b26021403070290f93a5c/5851918a0626310a2c186a5e.Google Scholar
• Friedman JH (2001) Greedy function approximation: A gradient boosting machine. Ann. Statist. 29(5):1189–1232.Crossref, Google Scholar
• Fu J, Kumar A, Soh M, Levine S (2019) Diagnosing bottlenecks in deep q-learning algorithms. Chaudhuri K, Salakhutdinov R, eds. Proc. 36th Internat. Conf. Machine Learn., vol. 97, Proceedings of Machine Learning Research Series (PMLR), 2021–2030.Google Scholar
• Goli A, Reiley D, Zhang H (2021) Personalized versioning: Product strategies constructed from experiments on pandora. Preprint, submitted July 8, last revised September 29, https://dx.doi.org/10.2139/ssrn.3874243.Google Scholar
• Gordon GJ (1995) Stable function approximation in dynamic programming. Machine Learning Proc. (Elsevier, New York), 261–268.Crossref, Google Scholar
• Han S, Jung J, Wetherall D (2012) A study of third-party tracking by mobile apps in the wild. Technical report UW-CSE-12-03-01, University of Washington, Seattle.Google Scholar
• Hasselt H (2010) Double q-learning. Lafferty J, Williams C, Shawe-Taylor J, Zemel R, Culotta A, eds. Adv. Neural Inform. Processing Systems, vol. 23 (Curran Associates, Inc., Red Hook, NY), 2613–2621.Google Scholar
• Horsky D (1977) An empirical analysis of the optimal advertising policy. Management Sci. 23(10):1037–1049.Link, Google Scholar
• IAB (2021) 2020/2021 IAB internet advertising revenue report. Accessed April 7, 2021, https://www.iab.com/insights/internet-advertising-revenue-report/.Google Scholar
• Jeziorski P, Segal I (2015) What makes them click: Empirical analysis of consumer demand for search advertising. Amer. Econom. J. Microeconom. 7(3):24–53.Crossref, Google Scholar
• Kallus N, Uehara M (2020) Double reinforcement learning for efficient off-policy evaluation in Markov decision processes. J. Machine Learn. Res. 21:167–1.Google Scholar
• Kar W, Swaminathan V, Albuquerque P (2015) Selection and ordering of linear online video ads. Werthner H, Zanker M, conference chairs. Proc. 9th ACM Conf. on Recommender Systems (ACM, New York), 203–210.Google Scholar
• Kempe D, Mahdian M (2008) A cascade model for externalities in sponsored search. Papadimitriou C, Zhang S, eds. Proc. Internat. Workshop on Internet and Network Econom. (Springer, Berlin), 585–596.Google Scholar
• Kristianto D (2021) Winning the attention war: Consumers in nine major markets now spend more than four hours a day in apps. Accessed April 8, 2021, https://www.appannie.com/en/insights/market-data/q1-2021-market-index/.Google Scholar
• Le H, Voloshin C, Yue Y (2019) Batch policy learning under constraints. Chaudhuri K, Salakhutdinov R, eds. Proc. 36th Internat. Conf. on Machine Learn. (PMLR), 3703–3712.Google Scholar
• Lee K, Laskin M, Srinivas A, Abbeel P (2021) Sunrise: A simple unified framework for ensemble learning in deep reinforcement learning. Meila M, Zhang T, eds. Proc. 38th Internat. Conf. on Machine Learn. (PMLR), 6131–6141.Google Scholar
• Levine S, Kumar A, Tucker G, Fu J (2020) Offline reinforcement learning: Tutorial, review, and perspectives on open problems. Preprint, submitted May 4, https://arxiv.org/abs/2005.01643.Google Scholar
• Ling X, Deng W, Gu C, Zhou H, Li C, Sun F (2017) Model ensemble for click prediction in Bing search ads. Barrett R, Cummings R, chairs. Proc. 26th Internat. Conf. on World Wide Web Companion (International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE), 689–698.Google Scholar
• Little JD (1979) Aggregate advertising models: The state of the art. Oper. Res. 27(4):629–667.Link, Google Scholar
• Lu S, Yang S (2017) Investigating the spillover effect of keyword market entry in sponsored search advertising. Marketing Sci. 36(6):976–998.Link, Google Scholar
• Manchanda P, Dubé J-P, Goh KY, Chintagunta PK (2006) The effect of banner advertising on Internet purchasing. J. Marketing Res. 43(1):98–108.Crossref, Google Scholar
• Mandel T, Liu Y-E, Levine S, Brunskill E, Popovic Z (2014) Offline policy evaluation across representations with applications to educational games. Bazzan A, Huhns M, chairs. Proc. Internat. Conf. on Autonomous Agents and Multi-Agent Systems (International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC), 1077–1084.Google Scholar
• Mannor S, Simester D, Sun P, Tsitsiklis JN (2007) Bias and variance approximation in value function estimates. Management Sci. 53(2):308–322.Link, Google Scholar
• Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, et al. (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533.Crossref, Google Scholar
• Mullainathan S, Spiess J (2017) Machine learning: An applied econometric approach. J. Econom. Perspective 31(2):87–106.Crossref, Google Scholar
• Nahum-Shani I, Smith SN, Spring BJ, Collins LM, Witkiewitz K, Tewari A, Murphy SA (2017) Just-in-time adaptive interventions (JITAIs) in mobile health: Key components and design principles for ongoing health behavior support. Ann. Behav. Medicine 52(6):446–462.Crossref, Google Scholar
• Naik PA, Mantrala MK, Sawyer AG (1998) Planning media schedules in the presence of dynamic advertising quality. Marketing Sci. 17(3):214–235.Link, Google Scholar
• Nerlove M, Arrow KJ (1962) Optimal advertising policy under dynamic conditions. Economica 29(114):129–142.Crossref, Google Scholar
• Rafieian O (2020) Revenue-optimal dynamic auctions for adaptive ad sequencing. Working paper, Cornell Tech, New York.Google Scholar
• Rafieian O, Yoganarasimhan H (2021) Targeting and privacy in mobile advertising. Marketing Sci. 40(2):193–218.Link, Google Scholar
• Rafieian O, Yoganarasimhan H (2022a) AI and personalization. Preprint, submitted June 10, https://dx.doi.org/10.2139/ssrn.4123356.Google Scholar
• Rafieian O, Yoganarasimhan H (2022b) Variety effects in mobile advertising. J. Marketing Res. 59(4):718–738.Crossref, Google Scholar
• Rosenbaum PR, Rubin DB (1983) The central role of the propensity score in observational studies for causal effects. Biometrika 70(1):41–55.Crossref, Google Scholar
• Rossi PE, McCulloch RE, Allenby GM (1996) The value of purchase history data in target marketing. Marketing Sci. 15(4):321–340.Link, Google Scholar
• Rutz OJ, Bucklin RE (2011) From generic to branded: A model of spillover in paid search advertising. J. Marketing Res. 48(1):87–102.Crossref, Google Scholar
• Sahni NS (2015) Effect of temporal spacing between advertising exposures: Evidence from online field experiments. Quant. Marketing Econom. 13(3):203–247.Crossref, Google Scholar
• Samuel AL (1959) Some studies in machine learning using the game of checkers. IBM J. Res. Development 3(3):210–229.Crossref, Google Scholar
• Sawyer AG, Ward S (1979) Carry-over effects in advertising communication. Res. Marketing 2:259–314.Google Scholar
• Schwartz EM, Bradlow ET, Fader PS (2017) Customer acquisition via display advertising using multi-armed bandit experiments. Marketing Sci. 36(4):500–522.Link, Google Scholar
• Simester DI, Sun P, Tsitsiklis JN (2006) Dynamic catalog mailing policies. Management Sci. 52(5):683–696.Link, Google Scholar
• Simon H (1982) ADPULS: An advertising model with wearout and pulsation. J. Marketing Res. 19(3):352–363.Crossref, Google Scholar
• Sun Z, Dawande M, Janakiraman G, Mookerjee V (2017) Not just a fad: Optimal sequencing in mobile in-app advertising. Inform. Systems Res. 28(3):511–528.Link, Google Scholar
• Sutton RS, Barto AG (2018) Reinforcement Learning: An Introduction (MIT Press, Cambridge, MA).Google Scholar
• Tellis GJ (2003) Effective Advertising: Understanding When, How, and Why Advertising Works (Sage Publications, Thousand Oaks, CA).Google Scholar
• Theocharous G, Thomas PS, Ghavamzadeh M (2015) Personalized ad recommendation systems for life-time value optimization with guarantees. Yang Q, Wooldridge M, eds. Proc. 24th Internat. Joint Conf. on Artificial Intelligence (AAAI Press, Menlo Park, CA), 1806–1812.Google Scholar
• Thomas P, Brunskill E (2016) Data-efficient off-policy policy evaluation for reinforcement learning. Balcan MF, Weinberger KQ, eds. Proc. Internat. Conf. on Machine Learn. (PMLR), 2139–2148.Google Scholar
• Thomas P, Theocharous G, Ghavamzadeh M (2015) High-confidence off-policy evaluation. Proc. AAAI Conf. on Artificial Intelligence, vol. 29 (AAAI Press, Menlo Park, CA), 3000–3006.Google Scholar
• Thomas PS, da Silva BC, Barto AG, Giguere S, Brun Y, Brunskill E (2019) Preventing undesirable behavior of intelligent machines. Science 366(6468):999–1004.Crossref, Google Scholar
• Tsitsiklis JN, Van Roy B (1996) Feature-based methods for large scale dynamic programming. Machine Learn. 22(1):59–94.Crossref, Google Scholar
• Urban GL, Liberali G, MacDonald E, Bordley R, Hauser JR (2013) Morphing banner advertising. Marketing Sci. 33(1):27–46.Link, Google Scholar
• Van Hasselt H, Doron Y, Strub F, Hessel M, Sonnerat N, Modayil J (2018) Deep reinforcement learning and the deadly triad. Preprint, submitted December 6, https://arxiv.org/abs/1812.02648.Google Scholar
• Wilbur KC (2008) A two-sided, empirical model of television advertising and viewing markets. Marketing Sci. 27(3):356–378.Link, Google Scholar
• Wilbur KC, Xu L, Kempe D (2013) Correcting audience externalities in television advertising. Marketing Sci. 32(6):892–912.Link, Google Scholar
• Yi J, Chen Y, Li J, Sett S, Yan TW (2013) Predictive model performance: Offline and online evaluations. Ghani R, Senator TE, Bradley P, Parekh R, He J, eds. Proc. 19th ACM SIGKDD Internat. Conf. on Knowledge Discovery and Data Mining (ACM, New York), 1294–1302.Google Scholar
• Yoganarasimhan H (2020) Search personalization using machine learning. Management Sci. 66(3):1045–1070.Link, Google Scholar
• Yoganarasimhan H, Barzegary E, Pani A (2022) Design and evaluation of personalized free trials. Management Sci., ePub ahead of print August 10, https://doi.org/10.1287/mnsc.2022.4507.Google Scholar
• Zantedeschi D, Feit EM, Bradlow ET (2017) Measuring multichannel advertising response. Management Sci. 63(8):2706–2728.Link, Google Scholar