Customer Acquisition via Explainable Deep Reinforcement Learning

Effective customer acquisition heavily hinges on sequential targeting to ensure that appropriate marketing messages reach customers. Sequential targeting could guide customers through the acquisition process and thus, optimize long-term revenue for the firm. Toward this goal, reinforcement learning (RL) has demonstrated great potential in facilitating sequential targeting during user acquisition. However, decisions made by RL during this process often lack explainability. We introduce the deep recurrent Q-network with attention model, which optimizes the long-term reward of sequential targeting while enhancing the explainability of the decisions. The key idea of the proposed model is to revise Q-learning by adding an attention mechanism to create a bottleneck, forcing the model to focus on features of the next ad exposure that will lead to optimal long-term rewards. We estimate our model using a comprehensive data set from a digital bank. The empirical results show that the proposed model is explainable and also outperforms state-of-the-art methods in terms of long-term revenue optimization. Specifically, the attention mechanism within the model functions as forward planning. The forward planning can spot those features in the next ad exposure that are more likely to lead to the optimal outcome. We further demonstrate how the model makes targeting decisions of advertising channel choices by showing that the model can (1) learn optimal ad channels to target customers from different industries, (2) adjust advertising channels in response to dynamic customer behaviors, and (3) learn the seasonality of the customer’s industry and calibrate the ad channel correspondingly.

History: Olivia Liu Sheng, Senior Editor; Huimin Zhao, Associate Editor.

Funding: The authors thank General Research Fund [Grant 16500022] of Hong Kong Research Grants Council, [Grant RMGS20EG01-F] from HKUST, and [Grant WEB19EG01-F] for supporting this research.

Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2022.0529.