Reinforcement Learning for Intensity Control: An Application to Choice-Based Network Revenue Management

References

• Adelman D (2007) Dynamic bid prices in revenue management. Oper. Res. 55(4):647–661.Link, Google Scholar
• Azagirre X, Balwally A, Candeli G, Chamandy N, Han B, King A, Lee H, et al. (2024) A better match for drivers and riders: Reinforcement learning at Lyft. INFORMS J. Appl. Anal. 54(1):71–83.Link, Google Scholar
• Bradtke SJ, Duff MO (1995) Reinforcement learning methods for continuous-time Markov decision problems. Adv. Neural Inform. Processing Systems 7:393–400. Google Scholar
• Brémaud P (1981) Point Processes and Queues: Martingale Dynamics (Springer-Verlag, New York).Crossref, Google Scholar
• Chen N, Hu M (2023) Frontiers in service science: Data-driven revenue management: The interplay of data, model, and decisions. Service Sci. 15(2):79–91.Link, Google Scholar
• Chen H, Yao DD (1990) Optimal intensity control of a queueing system with state-dependent capacity limit. IEEE Trans. Automatic Control 35(4):459–464.Crossref, Google Scholar
• Dai JG, Gluzman M (2022) Queueing network controls via deep reinforcement learning. Stochastic Systems 12(1):30–67.Link, Google Scholar
• Dai M, Dong Y, Jia Y (2023) Learning equilibrium mean-variance strategy. Math. Finance 33(4):1166–1212.Crossref, Google Scholar
• Das TK, Gosavi A, Mahadevan S, Marchalleck N (1999) Solving semi-Markov decision problems using average reward reinforcement learning. Management Sci. 45(4):560–574.Link, Google Scholar
• Gallego G, Topaloglu H (2019) Revenue Management and Pricing Analytics (Springer, New York).Crossref, Google Scholar
• Gallego G, van Ryzin G (1997) A multiproduct dynamic pricing problem and its applications to network yield management. Oper. Res. 45(1):24–41.Link, Google Scholar
• Gallego G, Iyengar G, Phillips R, Dubey A (2004) Managing flexible products on a network. CORC Technical Report No. TR-2004-01, Department of Industrial Engineering and Operations Research, Columbia University, New York.Google Scholar
• Gao X, Zhou XY (2024) Logarithmic regret bounds for continuous-time average-reward Markov decision processes. SIAM J. Control Optim. 62(5):2529–2556.Crossref, Google Scholar
• Gao X, Zhou X (2026) Square-root regret bounds for continuous-time episodic Markov decision processes. Math. Oper. Res. 51(1):333–357.Link, Google Scholar
• Gao X, Li L, Zhou XY (2026) Reinforcement learning for jump-diffusions, with financial applications. Math. Finance, ePub ahead of print March 17, https://doi.org/10.1111/mafi.70027.Crossref, Google Scholar
• Gijsbrechts J, Boute RN, Van Mieghem JA, Zhang DJ (2022) Can deep reinforcement learning improve inventory management? Performance on lost sales, dual-sourcing, and multi-echelon problems. Manufacturing Service Oper. Management 24(3):1349–1368.Link, Google Scholar
• Guo X, Hernández-Lerma O (2009) Continuous-Time Markov Decision Processes: Theory and Applications (Springer, Berlin).Crossref, Google Scholar
• Guo X, Xu R, Zariphopoulou T (2022) Entropy regularization for mean field games with learning. Math. Oper. Res. 47(4):3239–3260.Link, Google Scholar
• Haarnoja T, Zhou A, Abbeel P, Levine S (2018) Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. Dy J, Krause A, eds. Proc. 35th Internat. Conf. Machine Learn., vol. 80 (PMLR, New York), 1861–1870.Google Scholar
• Jia Y, Zhou XY (2022a) Policy evaluation and temporal-difference learning in continuous time and space: A martingale approach. J. Machine Learn. Res. 23(154):1–55.Google Scholar
• Jia Y, Zhou XY (2022b) Policy gradient and actor-critic learning in continuous time and space: Theory and algorithms. J. Machine Learn. Res. 23(275):1–50.Google Scholar
• Jia Y, Zhou XY (2023) q-learning in continuous time. J. Machine Learn. Res. 24(161):1–61.Google Scholar
• Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. Bengio Y, LeCun Y, eds. Proc. 3rd Internat. Conf. Learn. Representations (ICLR, Appleton, WI).Google Scholar
• Konda VR, Tsitsiklis JN (1999) Actor-critic algorithms. Adv. Neural Inform. Processing Systems 12:1008–1014.Google Scholar
• Li T, Wang C, Wang Y, Tang S, Chen N (2026) Deep reinforcement learning for online assortment customization: A data-driven approach. Production Oper. Management 35(2):665–684.Crossref, Google Scholar
• Liu Q, van Ryzin G (2008) On the choice-based linear programming model for network revenue management. Manufacturing Service Oper. Management 10(2):288–310.Link, Google Scholar
• Ma Y, Rusmevichientong P, Sumida M, Topaloglu H (2020) An approximation algorithm for network revenue management under nonstationary arrivals. Oper. Res. 68(3):834–855.Link, Google Scholar
• Mnih V, Badia AP, Mirza M, Graves A, Lillicrap TP, Harley T, Silver D, Kavukcuoglu K (2016) Asynchronous methods for deep reinforcement learning. Balcan MF, Weinberger KQ, eds. Proc. 33rd Internat. Conf. Machine Learn., vol. 48 (PMLR, New York), 1928–1937.Google Scholar
• Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, et al. (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533.Crossref, Google Scholar
• Oroojlooyjadid A, Nazari M, Snyder LV, Takáč M (2022) A deep Q-network for the beer game: Deep reinforcement learning for inventory optimization. Manufacturing Service Oper. Management 24(1):285–304.Link, Google Scholar
• Puterman ML (2014) Markov Decision Processes: Discrete Stochastic Dynamic Programming (John Wiley & Sons, Hoboken, NJ).Google Scholar
• Robbins H, Monro S (1951) A stochastic approximation method. Ann. Math. Statist. 22(3):400–407.Crossref, Google Scholar
• Sinclair SR, Banerjee S, Yu CL (2023) Adaptive discretization in online reinforcement learning. Oper. Res. 71(5):1636–1652.Link, Google Scholar
• Sinclair SR, Wang T, Jain G, Banerjee S, Yu CL (2020) Adaptive discretization for model-based reinforcement learning. Adv. Neural Inform. Processing Systems 33:3858–3871.Google Scholar
• Strauss AK, Klein R, Steinhardt C (2018) A review of choice-based revenue management: Theory and methods. Eur. J. Oper. Res. 271(2):375–387.Crossref, Google Scholar
• Sutton RS, Barto AG (2018) Reinforcement Learning: An Introduction, 2nd ed. (MIT Press, Cambridge, MA).Google Scholar
• Tallec C, Blier L, Ollivier Y (2019) Making deep Q-learning methods robust to time discretization. Chaudhuri K, Salakhutdinov R, eds. Proc. 36th Internat. Conf. Machine Learn., vol. 97 (PMLR, New York), 6096–6104.Google Scholar
• Talluri K, van Ryzin G (2004) Revenue management under a general discrete choice model of consumer behavior. Management Sci. 50(1):15–33.Link, Google Scholar
• Wang H, Zhou XY (2020) Continuous-time mean–variance portfolio selection: A reinforcement learning framework. Math. Finance 30(4):1273–1308.Crossref, Google Scholar
• Wang B, Gao X, Li L (2026) Reinforcement learning for continuous-time optimal execution: Actor-critic algorithm and error analysis. Finance Stoch. 30(2):597–655.Crossref, Google Scholar
• Wang H, Zariphopoulou T, Zhou XY (2020) Reinforcement learning in continuous time and space: A stochastic control approach. J. Machine Learn. Res. 21(198):1–34.Google Scholar
• Wu B, Li L (2024) Reinforcement learning for continuous-time mean-variance portfolio selection in a regime-switching market. J. Econom. Dynam. Control 158:104787.Crossref, Google Scholar
• Zhalechian M, Keyvanshokooh E, Shi C, Van Oyen MP (2023) Data-driven hospital admission control: A learning approach. Oper. Res. 71(6):2111–2129.Link, Google Scholar
• Zhang D (2011) An improved dynamic programming decomposition approach for network revenue management. Manufacturing Service Oper. Management 13(1):35–52.Link, Google Scholar
• Zhang D, Adelman D (2009) An approximate dynamic programming approach to network revenue management with customer choice. Transportation Sci. 43(3):381–394.Link, Google Scholar
• Zhang D, Cooper WL (2005) Revenue management for parallel flights with customer-choice behavior. Oper. Res. 53(3):415–431.Link, Google Scholar
• Zhao H, Tang W, Yao DD (2023) Policy optimization for continuous reinforcement learning. Adv. Neural Inform. Processing Systems 36:13637–13663.Crossref, Google Scholar