On the Taylor Expansion of Value Functions

References

• Altman E (1993) Asymptotic properties of constrained Markov decision processes. Zeitschrift für Oper. Res. 37(2):151–170.Google Scholar
• Ata B, Gurvich I (2012) On optimality gaps in the Halfin–Whitt regime. Ann. Appl. Probab. 22(1):407–455.Crossref, Google Scholar
• Bertsekas DP (2007) Approximate Dynamic Programming, Dynamic Programming and Optimal Control, vol. 2, 3rd ed. (Athena Scientific, Belmont, MA).Google Scholar
• Bertsekas DP (2011) Approximate policy iteration: A survey and some new methods. J. Control Theory Appl. 9(3):310–335.Crossref, Google Scholar
• Bertsekas DP (2019) Feature-based aggregation and deep reinforcement learning: A survey and some new implementations. IEEE/CAA J. Automatica Sinica 6(1):1–31.Google Scholar
• Bertsekas DP, Tsitsiklis JN (1996) Neuro-Dynamic Programming (Athena Scientific, Belmont, MA).Google Scholar
• Borkar V, Budhiraja A (2004) Ergodic control for constrained diffusions: Characterization using HJB equations. SIAM J. Control Optim. 43(4):1467–1492.Crossref, Google Scholar
• Braverman A, Dai JG (2017) Stein’s method for steady-state diffusion approximations of M/Ph/n+M systems. Ann. Appl. Probab. 27(1):550–581.Crossref, Google Scholar
• Chen W, Huang D, Kulkarni AA, Unnikrishnan J, Zhu Q, Mehta P, Meyn S, Wierman A (2009) Approximate dynamic programming using fluid and diffusion approximations with applications to power management. Proc. 48th IEEE Conf. Decision Control (IEEE, Piscataway, NJ), 3575–3580.Google Scholar
• Dai JG, Shi P (2017) A two-time-scale approach to time-varying queues in hospital inpatient flow management. Oper. Res. 65(2):514–536.Link, Google Scholar
• Dupuis PG, Ishii H (1990) On oblique derivative problems for fully nonlinear second-order elliptic partial differential equations on nonsmooth domains. Nonlinear Anal.: Theory Methods Appl. 15(12):1123–1138.Crossref, Google Scholar
• Dupuis PG, James MR (1998) Rates of convergence for approximation schemes in optimal control. SIAM J. Control Optim. 36(2):719–741.Crossref, Google Scholar
• Gilbarg D, Trudinger NS (2001) Elliptic Partial Differential Equations of Second Order (Springer-Verlag, New York).Crossref, Google Scholar
• Gurvich I (2014) Diffusion models and steady-state approximations for exponentially ergodic Markovian queues. Ann. Appl. Probab. 24(6):2527–2559.Crossref, Google Scholar
• Harrison JM (2013) Brownian Motion and Stochastic Flow Systems (Cambridge University Press, New York).Google Scholar
• Huang J, Gurvich I (2018) Beyond heavy-traffic regimes: Universal bounds and controls for the single-server queue. Oper. Res. 66(4):1168–1188.Link, Google Scholar
• Koçağa YL, Ward AR (2010) Admission control for a multi-server queue with abandonment. Queueing Systems 65(3):275–323.Crossref, Google Scholar
• Kushner H, Dupuis PG (2013) Numerical Methods for Stochastic Control Problems in Continuous Time, vol. 24 (Springer-Verlag, New York).Google Scholar
• Larsson S, Thomée V (2008) Partial Differential Equations with Numerical Methods, vol. 45 (Springer-Verlag, Berlin, Heidelberg).Google Scholar
• Lieberman GM (2013) Oblique Derivative Problems for Elliptic Equations (World Scientific, Hackensack, NJ).Crossref, Google Scholar
• McShane EJ (1934) Extension of range of functions. Bull. Amer. Math. Soc. 40(12):837–842.Crossref, Google Scholar
• Moallemi C, Kumar S, Van Roy B (2008) Approximate and data-driven dynamic programming for queueing networks. Working paper, Stanford University, CA.Google Scholar
• Powell WB (2007) Approximate Dynamic Programming: Solving the Curses of Dimensionality (John Wiley & Sons, Hoboken, NJ).Google Scholar
• Weerasinghe A, Mandelbaum A (2013) Abandonment vs. blocking in many-server queues: Asymptotic optimality in the QED regime. Queueing Systems 75(2):279–337.Crossref, Google Scholar
• Whitt W (2002) Stochastic-Process Limits: An Introduction to Stochastic-Process Limits and Their Application to Queues (Springer-Verlag, New York).Crossref, Google Scholar
• Zhang BZ, Gurvich I (2018) Aggregation via local moment matching. Working paper, Cornell University, Ithaca, NY.Google Scholar