Drift Control of High-Dimensional Reflected Brownian Motion: A Computational Method Based on Neural Networks

References

• Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, et al. (2016) Tensorflow: A system for large-scale machine learning. OSDI, Savannah, GA, vol. 16 (USENIX Association, Berkeley, CA), 265–283.Google Scholar
• Andradóttir S, Heyman DP, Ott TJ (1993) Variance reduction through smoothing and control variates for Markov Chain simulations. ACM Trans. Model. Comput. Simulation 3(3):167–189.Google Scholar
• Ata B (2006) Dynamic control of a multiclass queue with thin arrival streams. Oper. Res. 54(5):876–892.Link, Google Scholar
• Ata B, Barjesteh N (2023) An approximate analysis of dynamic pricing, outsourcing, and scheduling policies for a multiclass make-to-stock queue in the heavy traffic regime. Oper. Res. 71(1):341–357.Link, Google Scholar
• Ata B, Kasikaralar E (2023) Dynamic scheduling of a multiclass queue in the Halfin-Whitt regime: A computational approach for high-dimensional problems. Preprint, submitted November 29, https://arxiv.org/abs/2311.18128.Google Scholar
• Ata B, Zhou Y (2024) Analysis and improvement of eviction enforcement. Working paper, University of Chicago, Chicago.Google Scholar
• Ata B, Harrison JM, Shepp LA (2005) Drift rate control of a Brownian processing system. Ann. Appl. Probab. 15(2):1145–1160.Google Scholar
• Ata B, Harrison JM, Si N (2024) Singular control of (reflected) Brownian motion: A computational method suitable for queueing applications. Queueing Systems, 1–37.Google Scholar
• Ata B, Lee D, Sonmez E (2019) Dynamic volunteer staffing in multicrop gleaning operations. Oper. Res. 67(2):295–314.Abstract, Google Scholar
• Bar-Ilan A, Marion NP, Perry D (2007) Drift control of international reserves. J. Econom. Dynam. Control 31:3110–3137.Google Scholar
• Beck C, Hutzenthaler M, Jentzen A, Kuckuck B (2023) An overview on deep learning-based approximation methods for partial differential equations. Discrete Continuous Dynamic. Systems Ser. B 28(6):3697–3746.Google Scholar
• Billingsley P (1999) Convergence of Probability Measures, 2nd ed. (John Wiley & Sons, Hoboken, NJ).Google Scholar
• Blanchet J, Chen X, Si N, Glynn PW (2021) Efficient steady-state simulation of high-dimensional stochastic networks. Stochastic Systems 11(2):174–192.Link, Google Scholar
• Borkar V, Budhiraja A (2005) Ergodic control for constrained diffusions: Characterization using HJB equations. SIAM J. Control Optim. 43(4):1467–1492.Google Scholar
• Budhiraja A, Lee C (2007) Long time asymptotics for controlled diffusions in polyhedral domains. Stochastic Processes Their Appl. 117(8):1014–1036.Google Scholar
• Çelik S, Maglaras C (2008) Dynamic pricing and lead-time quotation for a multiclass make-to-order queue. Management Sci. 54(6):1132–1146.Link, Google Scholar
• Dai JG, Gluzman M (2022) Queueing network controls via deep reinforcement learning. Stochastic Systems 12(1):30–67.Link, Google Scholar
• Dai JG, Harrison JM (1991) Steady-state analysis of RBM in a rectangle: Numerical methods and a queueing application. Ann. Appl. Probab. 1(1):16–35.Google Scholar
• Dai JG, Williams R (1996) Existence and uniqueness of semimartingale reflecting Brownian motions in convex polyhedrons. Theory Probab. Appl. 40(1):1–40.Google Scholar
• Dupuis P, Ishii H (1991) On oblique derivative problems for fully nonlinear second-order elliptic PDEs on domains with corners. Hokkaido Math. J. 20:135–164.Google Scholar
• E W, Han J, Jentzen A (2022) Algorithms for solving high dimensional PDEs: From nonlinear Monte Carlo to machine learning. Nonlinearity 35:278–310.Google Scholar
• Ghosh AP, Weerasinghe AP (2007) Optimal buffer size for a stochastic processing network in heavy traffic. Queueing Systems 55(3):147–159.Google Scholar
• Ghosh AP, Weerasinghe AP (2010) Optimal buffer size and dynamic rate control for a queueing system with impatient customers in heavy traffic. Stochastic Processes Their Appl. 120(11):2103–2141.Google Scholar
• Han J, Long J (2020) Convergence of the deep BSDE method for coupled FBSDEs. Probab. Uncertainty Quantitative Risk 5(1):5.Google Scholar
• Han J, Jentzen A, Weinan E (2018) Solving high-dimensional partial differential equations using deep learning. Proc. Natl. Acad. Sci. USA 115(34):8505–8510.Google Scholar
• Harrison JM (1988) Brownian models of queueing networks with heterogeneous customer populations. Fleming W, Lions PL, eds. Stochastic Differential Systems, Stochastic Control Theory and Applications, The IMA Volumes in Mathematics and Its Applications, vol. 10 (Springer, New York), 147–186.Google Scholar
• Harrison JM (2000) Brownian models of open processing networks: Canonical representation of workload. Ann. Appl. Probab. 10(1):75–103.Google Scholar
• Harrison JM (2013) Brownian Models of Performance and Control (Cambridge University Press, Cambridge, UK).Google Scholar
• Harrison JM, Nguyen V (1993) Brownian models of multiclass queueing networks: Current status and open problems. Queueing Systems 13:5–40.Google Scholar
• Harrison JM, Reiman MI (1981) Reflected Brownian motion on an orthant. Ann. Probab. 9(2):302–308.Google Scholar
• Harrison JM, Wein LM (1989) Scheduling networks of queues: Heavy traffic analysis of a simple open network. Queueing Systems 5:265–279.Google Scholar
• Harrison JM, Wein LM (1990) Scheduling networks of queues: Heavy traffic analysis of a two-station closed network. Oper. Res. 38(6):1052–1064.Link, Google Scholar
• Harrison JM, Williams RJ (1987) Brownian models of open queueing networks with homogeneous customer populations. Stochastics 22(2):77–115.Google Scholar
• Henderson SG, Glynn PW (2002) Approximating martingales for variance reduction in Markov process simulation. Math. Oper. Res. 27(2):253–271.Link, Google Scholar
• Hochreiter S (1998) The vanishing gradient problem during learning recurrent neural nets and problem solutions. Internat. J. Uncertainty Fuzziness Knowledge-Based Systems 6(02):107–116.Google Scholar
• Iglehart DL, Whitt W (1970a) Multiple channel queues in heavy traffic. I. Adv. Appl. Probab. 2(1):150–177.Google Scholar
• Iglehart DL, Whitt W (1970b) Multiple channel queues in heavy traffic. II. Sequences, networks, and batches. Adv. Appl. Probab. 2(2):355–369.Google Scholar
• Karatzas I (1983) A class of singular control problems. Adv. Appl. Probab. 15(2):225–254.Google Scholar
• Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. Preprint, submitted December 22, https://arxiv.org/abs/1412.6980.Google Scholar
• Krichagina EV, Taksar MI (1992) Diffusion approximation for GI/G/1 controlled queues. Queueing Systems 12:333–367.Google Scholar
• Kushner HJ (2001) Heavy Traffic Analysis of Controlled Queueing and Communication Networks, Stochastic Modelling and Applied Probability, vol. 28 (Springer, New York).Google Scholar
• Kushner HJ, Martins LF (1991) Numerical methods for stochastic singular control problems. SIAM J. Control Optim. 29(6):1443–1475.Google Scholar
• Martins LF, Kushner HJ (1990) Routing and singular control for queueing networks in heavy traffic. SIAM J. Control Optim. 28(5):1209–1233.Google Scholar
• Martins LF, Shreve SE, Soner HM (1996) Heavy traffic convergence of a controlled, multiclass queueing system. SIAM J. Control Optim. 34(6):2133–2171.Google Scholar
• Oksendal B (2003) Stochastic Differential Equations: An Introduction with Applications, 6th ed. (Springer Science & Business Media, New York).Google Scholar
• Ormeci Matoglu LM, Vande Vate JH (2011) Drift control with changeover costs. Oper. Res. 59(2):427–439.Link, Google Scholar
• Peterson WP (1991) A heavy traffic limit theorem for networks of queues with multiple customer types. Math. Oper. Res. 16(1):90–118.Link, Google Scholar
• Rasamoelina AD, Adjailia F, Sinčák P (2020) A review of activation function for artificial neural network. 2020 IEEE 18th World Sympos. Appl. Machine Intelligence Informatics (SAMI) (IEEE, Piscataway, NJ), 281–286.Google Scholar
• Reiman MI (1984) Open queueing networks in heavy traffic. Math. Oper. Res. 9(3):441–458.Link, Google Scholar
• Rubino M, Ata B (2009) Dynamic control of a make-to-order, parallel-server system with cancellations. Oper. Res. 57(1):94–108.Link, Google Scholar
• Taylor LM, Williams RJ (1993) Existence and uniqueness of semimartingale reflecting Brownian motions in an orthant. Probab. Theory Related Fields 96(3):283–317.Google Scholar
• Vande Vate JH (2021) Average cost Brownian drift control with proportional changeover costs. Stochastic Systems 11(3):218–263.Link, Google Scholar
• Wein LM (1991) Brownian networks with discretionary routing. Oper. Res. 39(2):322–340.Link, Google Scholar
• Williams RJ (1996) On the approximation of queueing networks in heavy traffic. Stochastic Networks Theory Appl. 4:35–56.Google Scholar
• Williams RJ (1998a) An invariance principle for semimartingale reflecting Brownian motions in an orthant. Queueing Systems 30:5–25.Google Scholar
• Williams RJ (1998b) Diffusion approximations for open multiclass queueing networks: Sufficient conditions involving state space collapse. Queueing Systems 30:27–88.Google Scholar
• Winkelbauer A (2012) Moments and absolute moments of the normal distribution. Preprint, submitted September 19, https://arxiv.org/abs/1209.4340.Google Scholar
• Zhang KS, Peyré G, Fadili J, Pereyra M (2020) Wasserstein control of mirror Langevin Monte Carlo. Conf. Learn. Theory (PMLR, New York), 3814–3841.Google Scholar
• Zhou M, Han J, Lu J (2021a) Actor-critic method for high dimensional static Hamilton–Jacobi–Bellman partial differential equations based on neural networks. SIAM J. Sci. Comput. 43(6):A4043–A4066.Google Scholar
• Zhou M, Han J, Lu J (2021b) Code for “Actor-critic method for high dimensional static Hamilton–Jacobi–Bellman partial differential equations based on neural networks.” https://github.com/MoZhou1995/DeepPDE_ActorCritic.Google Scholar