Customer Acquisition via Explainable Deep Reinforcement Learning

References

• Agricultural Bank of China (2022) Agriculture seasonal loan. Accessed July 10, 2023, https://www.abchina.com/cn/RuralSvc/Businesses/jjdk/.Google Scholar
• Allen GI, Gan L, Zheng L (2023) Interpretable machine learning for discovery: Statistical challenges and opportunities. Annual Rev. Statist. Appl. 11:97–121.Google Scholar
• Annasamy RM, Sycara K (2019) Toward better interpretability in deep Q-networks. Proc. Conf. AAAI Artificial Intelligence 33(01):4561–4569.Crossref, Google Scholar
• AppsFlyer (2021) Fintech marketers invested $3B on user acquisition in 2020 according to Appsflyer. Accessed July 10, 2023, https://www.bloomberg.com/press-releases/2021-06-09/fintech-marketers-invested-3b-on-user-acquisition-in-2020-according-to-appsflyer.Google Scholar
• Berente N, Gu B, Recker J, Santhanam R (2021) Managing artificial intelligence. Management Inform. Systems Quart. 45(3):1433–1450.Google Scholar
• Bertsekas DP, Tsitsiklis JN (1995) Neuro-dynamic programming: An overview. Proc. 1995 34th IEEE Conf. Decision Control, vol. 1 (IEEE, Piscataway, NJ), 560–564.Google Scholar
• Brauwers G, Frasincar F (2021) A general survey on attention mechanisms in deep learning. IEEE Trans. Knowledge Data Engrg. 35(4):3279–3298.Crossref, Google Scholar
• Cao J, Leng Y (2021) Adaptive data acquisition for personalized recommender systems with optimality guarantees on short-form video platforms. Preprint, submitted May 11, http://dx.doi.org/10.2139/ssrn.3843086.Google Scholar
• Chaudhari S, Mithal V, Polatkan G, Ramanath R (2021) An attentive survey of attention models. ACM Trans. Intelligent Systems Tech. 12(5):1–32.Crossref, Google Scholar
• Chuck L (2022) The four pillars of customer acquisition strategy. Forbes (August 15), https://www.forbes.com/sites/forbesbusinesscouncil/2022/08/15/the-four-pillars-of-customer-acquisition-strategy/?sh=a84f6f63218d.Google Scholar
• Du M, Liu N, Hu X (2019) Techniques for interpretable machine learning. Comm. ACM 63(1):68–77.Crossref, Google Scholar
• Etchart M, Ladosz P, Mulvaney D (2019) Spatio-temporal attention deep recurrent Q-network for POMDPs. Proc. Progress Artificial Intelligence 19th EPIA Conf. Artificial Intelligence EPIA 2019 (Springer, Cham, Switzerland), 98–105.Google Scholar
• Fei H, Zhang Y, Ren Y, Ji D (2021) Optimizing attention for sequence modeling via reinforcement learning. IEEE Trans. Neural Networks Learn. Systems 33(8):3612–3621.Crossref, Google Scholar
• Fernández-Loría C, Provost F (2022) Causal decision making and causal effect estimation are not the same … and why it matters. INFORMS J. Data Sci. 1(1):4–16.Link, Google Scholar
• Frick TW, Belo R, Telang R (2023) Incentive misalignments in programmatic advertising: Evidence from a randomized field experiment. Management Sci. 69(3):1665–1686.Link, Google Scholar
• Hauser JR, Urban GL, Liberali G, Braun M (2009) Website morphing. Marketing Sci. 28(2):202–223.Link, Google Scholar
• Hausknecht M, Stone P (2015) Deep recurrent Q-learning for partially observable MDPs. 2015 AAAI Fall Sympos. Series (AAAI Press, Palo Alto, CA).Google Scholar
• He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. Proc. IEEE Conf. Comput. Vision Pattern Recognition (IEEE, Piscataway, NJ), 770–778.Google Scholar
• Hoffman DL, Novak TP (2000) How to acquire customers on the web. Harvard Bus. Rev. (May–June), https://hbr.org/2000/05/how-to-acquire-customers-on-the-web.Google Scholar
• ICO (2022) Draft anonymisation, pseudonymization and privacy enhancing technologies guidance. (December 31), https://ico.org.uk/about-the-ico/ico-and-stakeholder-consultations/ico-call-for-views-anonymisation-pseudonymisation-and-privacy-enhancing-technologies-guidance/.Google Scholar
• Itaya H, Hirakawa T, Yamashita T, Fujiyoshi H, Sugiura K (2021) Visual explanation using attention mechanism in actor-critic-based deep reinforcement learning. 2021 Internat. Joint Conf. Neural Networks (IJCNN) (IEEE, Piscataway, NJ), 1–10.Google Scholar
• Johnson G (2022) Economic research on privacy regulation: Lessons from the GDPR and beyond. Preprint, submitted December 8, http://dx.doi.org/10.2139/ssrn.4290849.Google Scholar
• Khanna M, Wimpey JS, Bruhn M, Singh S, Hommes M, Sorokina A (2017) MSME Finance Gap: Assessment of the Shortfalls and Opportunities in Financing Micro, Small, and Medium Enterprises in Emerging Markets (World Bank, Washington, DC).Google Scholar
• Kiran BR, Sobh I, Talpaert V, Mannion P, Al Sallab AA, Yogamani S, Pérez P (2022) Deep reinforcement learning for autonomous driving: A survey. IEEE Trans. Intelligent Transportation Systems 23(6):4909–4926.Crossref, Google Scholar
• Kokkodis M, Ipeirotis PG (2021) Demand-aware career path recommendations: A reinforcement learning approach. Management Sci. 67(7):4362–4383.Link, Google Scholar
• Kollnig K, Binns R, Van Kleek M, Lyngs U, Zhao J, Tinsman C, Shadbolt N (2021) Before and after GDPR: Tracking in mobile apps. Internet Policy Rev. 10(4).Crossref, Google Scholar
• Leng Y, Ruiz R, Liu X (2020) Geometric deep learning based recommender system and an interpretable decision support system. Preprint, submitted November 13, http://dx.doi.org/10.2139/ssrn.3696092.Google Scholar
• LSWZ (2022) Grain circulation data, national food and strategic reserves administration. Accessed July 10, 2023, http://www.lswz.gov.cn/html/ywpd/lstk/tj-sgsj.shtml.Google Scholar
• Lu T, Pál D, Pál M (2010) Contextual multi-armed bandits. Proc. Thirteenth Internat. Conf. Artificial Intelligence Statist. (PMLR, New York), 485–492.Google Scholar
• Mandel T, Liu YE, Brunskill E, Popović Z (2016) Offline evaluation of online reinforcement learning algorithms. Proc. AAAI Conf. Artificial Intelligence, vol. 30 (AAAI Press, Palo Alto, CA).Google Scholar
• Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, et al. (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533.Crossref, Google Scholar
• Mott A, Zoran D, Chrzanowski M, Wierstra D, Jimenez Rezende D (2019) Toward interpretable reinforcement learning using attention augmented agents. Adv. Neural Inform. Processing Systems 32(1107):12360–12369.Google Scholar
• Piatetsky-Shapiro G (1991) Discovery, analysis, and presentation of strong rules. Piatetsky-Shapiro G, Frawley W, eds. Knowledge Discovery in Databases (MIT Press, Cambridge, MA), 229–248.Google Scholar
• Puiutta E, Veith EM (2020) Explainable reinforcement learning: A survey. Internat. Cross-Domain Conf. Machine Learn. Knowledge Extraction (Springer, Cham, Switzerland), 77–95.Google Scholar
• Qi J, Zhou Q, Lei L, Zheng K (2021) Federated reinforcement learning: Techniques, applications, and open challenges. Preprint, submitted, August 26, https://arxiv.org/abs/2108.11887.Google Scholar
• Reiley D, Lewis RA, Schreiner T (2012) Ad attributes and attribution: Large-scale field experiments measure online customer acquisition. Preprint, submitted May 2, http://dx.doi.org/10.2139/ssrn.2049457.Google Scholar
• Schwartz EM, Bradlow ET, Fader PS (2017) Customer acquisition via display advertising using multi-armed bandit experiments. Marketing Sci. 36(4):500–522.Link, Google Scholar
• Serrano S, Smith NA (2019) Is attention interpretable? Korhonen A, Traum D, Màrquez L, eds. Proc. 57th Annual Meeting Assoc. Comput. Linguistics (Association for Computing Machinery, New York), 2931–2951.Google Scholar
• Shi W, Huang G, Song S, Wang Z, Lin T, Wu C (2020) Self-supervised discovering of interpretable features for reinforcement learning. IEEE Trans. Pattern Anal. Machine Intelligence 44(5):2712–2724.Google Scholar
• Simester D, Timoshenko A, Zoumpoulis SI (2020) Targeting prospective customers: Robustness of machine-learning methods to typical data challenges. Management Sci. 66(6):2495–2522.Link, Google Scholar
• Song Y, Sun T (2023) Ensemble experiments to optimize interventions along the customer journey: A reinforcement learning approach. Management Sci., ePub ahead of print October 4, https://doi.org/10.1287/mnsc.2023.4914.Link, Google Scholar
• Song Y, Sahoo N, Srinivasan S, Dellarocas C (2022) Uncovering characteristic response paths of a population. INFORMS J. Comput. 34(3):1661–1680.Link, Google Scholar
• Sutton RS, Barto AG (2018) Reinforcement Learning: An Introduction (MIT Press, Cambridge, MA).Google Scholar
• Tehrani P, Restuccia F, Levorato M (2021) Federated deep reinforcement learning for the distributed control of NextG wireless networks. 2021 IEEE Internat. Sympos. Dynamic Spectrum Access Networks (DySPAN) (IEEE, Piscataway, NJ), 248–253.Google Scholar
• Torrado RR, Bontrager P, Togelius J, Liu J, Perez-Liebana D (2018) Deep reinforcement learning for general video game AI. 2018 IEEE Conf. Comput. Intelligence Games (CIG) (IEEE, Piscataway, NJ), 1–8.Google Scholar
• Turek M (2018) Explainable artificial intelligence. Defense Advanced Research Projects Agency. Accessed July 10, 2023, https://www.darpa.mil/program/explainable-artificial-intelligence.Google Scholar
• Van Hasselt H, Guez A, Silver D (2016) Deep reinforcement learning with double Q-learning. Proc. AAAI Conf. Artificial Intelligence, vol. 30 (AAAI Press, Palo Alto, CA).Google Scholar
• Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. NIPS 2017:5998–6008.Google Scholar
• Wang T, He C, Jin F, Hu YJ (2022) Evaluating the effectiveness of marketing campaigns for malls using a novel interpretable machine learning model. Inform. Systems Res. 33(2):659–677.Link, Google Scholar
• Wang W, Li B, Luo X, Wang X (2023) Deep reinforcement learning for sequential targeting. Management Sci. 69(9):5439–5460.Link, Google Scholar
• Wang J, Zhang Y, Tang K, Wu J, Xiong Z (2019) Alphastock: A buying-winners-and-selling-losers investment strategy using interpretable deep reinforcement attention networks. Proc. 25th ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (Association for Computing Machinery, New York), 1900–1908.Google Scholar
• Wang Z, Schaul T, Hessel M, Hasselt H, Lanctot M, Freitas N (2016) Dueling network architectures for deep reinforcement learning. Internat. Conf. Machine Learn. (PMLR, New York), 1995–2003.Google Scholar
• World Bank (2019) Small and medium enterprises (SMES) finance. World Bank (October 16), https://www.worldbank.org/en/topic/smefinance.Google Scholar
• Watkins CJ, Dayan P (1992) Q-learning. Machine Learn. 8(3–4):279–292.Crossref, Google Scholar
• Zhang J, Curley SP (2018) Exploring explanation effects on consumers’ trust in online recommender agents. Internat. J. Human Comput. Interaction 34(5):421–432.Crossref, Google Scholar
• Zhang Q, Du Q, Liu G (2021) A whole-process interpretable and multi-modal deep reinforcement learning for diagnosis and analysis of Alzheimer’s disease. J. Neural Engrg. 18(6):066032.Crossref, Google Scholar