Learning to Optimally Stop Diffusion Processes, with Financial Applications

We study optimal stopping for diffusion processes with unknown model primitives within the continuous-time reinforcement learning (RL) framework Wang, Zariphopoulou and Zhou [Wang H, Zariphopoulou T, Zhou XY (2020), J. Machine Learn. Res. 21(198):1-34], and present applications to option pricing and portfolio choice. By penalizing the corresponding variational inequality formulation, we transform the stopping problem into a stochastic optimal control problem with two actions. We then randomize controls into Bernoulli distributions and add an entropy regularizer to encourage exploration. We derive a semianalytical optimal Bernoulli distribution, based on which we devise RL algorithms using the martingale approach Jia and Zhou [Jia Y, Zhou XY (2022a) J. Machine Learn. Res. 23(154):1–55]. We establish a policy improvement theorem and prove the fast convergence of the resulting policy iterations. We demonstrate the effectiveness of the algorithms in pricing finite-horizon American put options, solving Merton’s problem with transaction costs, and scaling to high-dimensional optimal stopping problems. In particular, we show that both the offline and online algorithms achieve high accuracy in learning the value functions and characterizing the associated free boundaries.

This paper has been accepted by Kay Giesecke for the Special Issue on AI for Finance and Business Decisions.

Funding: M. Dai was supported by the Hong Kong Research Grants Council [15212324, 15217123, 15213422, and T32-615/24-R], the Hong Kong Polytechnic University [P0042708, P0042456, and P0039114], and the Key Project of National Natural Science Foundation of China [72432005]. Y. Sun acknowledges financial support from a start-up fund at Peking University HSBC Business School. Z. Xu acknowledges financial support from The Hong Kong RGC [GRF 15204622, 15203423], NSFC 12571517, the PolyU-SDU Joint Research Center on Financial Mathematics, the CAS AMSS-PolyU Joint Laboratory of Applied Mathematics, the Research Centre for Quantitative Finance (1-CE03), and internal grants from The Hong Kong Polytechnic University. X. Zhou was supported by the Nie Center for Intelligent Asset Management at Columbia University. This work was also part of a Columbia-CityU/HK collaborative project that was supported by the InnoHK Initiative, the Government of the HKSAR, and the AIFT Lab.

Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2024.07614.