Robustness of Stochastic Optimal Control to Approximate Diffusion Models Under Several Cost Evaluation Criteria

References

• [1] Adams RA (1975) Sobolev Spaces (Academic Press, New York).Google Scholar
• [2] Almudevar A, Arruda EF (2012) Optimal approximation schedules for a class of iterative algorithms, with an application to multigrid value iteration. IEEE Trans. Automatic Control 57(12):3132–3146.Crossref, Google Scholar
• [3] Arapostathis A (2012) On the policy iteration algorithm for nondegenerate controlled diffusions under the ergodic criterion. Hernández-Hernández D, Minjárez-Sosa JA, eds. Optimization, Control, and Applications of Stochastic Systems (Birkhäuser, Boston), 1–12.Crossref, Google Scholar
• [4] Arapostathis A (2013) On the non-uniqueness of solutions to the average cost HJB for controlled diffusions with near-monotone costs. Preprint, submitted September 21, https://arxiv.org/abs/1309.6307.Google Scholar
• [5] Arapostathis A, Borkar VS, Ghosh MK (2012) Ergodic control of diffusion processes. Encyclopedia of Mathematics and its Applications, vol. 143 (Cambridge University Press, Cambridge, UK).Google Scholar
• [6] Arruda EF, Ourique F, Lacombe J, Almudevar A (2013) Accelerating the convergence of value iteration by using partial transition functions. Eur. J. Oper. Res. 229(1):190–198.Crossref, Google Scholar
• [7] Başar T, Bernhard P (1995) H-Infinity Optimal Control and Related Minimax Design Problems: A Dynamic Game Approach (Birkhäuser, Boston).Google Scholar
• [8] Bayraktar E, Dolinskyi L, Dolinsky Y (2020) Extended weak convergence and utility maximization with proportional transaction costs. Finance Stochastics 24:1013–1034.Crossref, Google Scholar
• [9] Benavoli A, Chisci L (2011) Robust stochastic control based on imprecise probabilities. IFAC Proc. Volumes 44(1):4606–4613.Crossref, Google Scholar
• [10] Bensoussan A, Lions JL (1984) Impulse Control and Quasi-Variational Inequalities (Gauthier-Villars, Bristol, UK).Google Scholar
• [11] Bhatt AG, Borkar VS (1996) Occupation measures for controlled Markov processes: Characterization and optimality. Ann. Probab. 24(3):1531–1562.Crossref, Google Scholar
• [12] Blanchet J, Murthy K (2016) Quantifying distributional model risk via optimal transport. Preprint, submitted April 6, https://dx.doi.org/10.2139/ssrn.2759640.Google Scholar
• [13] Boel RK, James MR, Petersen IR (2002) Robustness and risk-sensitive filtering. IEEE Trans. Automatic Control 47(3):451–461.Crossref, Google Scholar
• [14] Borkar VS (1986) A remark on the attainable distributions of controlled diffusions. Stochastics 18(1):17–23.Crossref, Google Scholar
• [15] Borkar VS (1989) Optimal Control of Diffusion Processes, Pitman Research Notes in Mathematics Series, vol. 203 (John Wiley & Sons, Inc., New York).Google Scholar
• [16] Borkar VS (1989) A topology for Markov controls. Appl. Math. Optim. 20:55–62.Crossref, Google Scholar
• [17] Borkar VS (2005) Controlled diffusion processes. Probab. Surveys 2:213–244.Crossref, Google Scholar
• [18] Borkar VS, Ghosh MK (1988) Ergodic control of multidimensional diffusions. I. The existence results. SIAM J. Control Optim. 26(1):112–126.Crossref, Google Scholar
• [19] Borkar VS, Ghosh MK (1990) Controlled diffusions with constraints. J. Math. Anal. Appl. 152(1):88–108.Crossref, Google Scholar
• [20] Borkar VS, Ghosh MK (1990) Ergodic control of multidimensional diffusions II. Adaptive control. Appl. Math. Optim. 21:191–220.Crossref, Google Scholar
• [21] Brezis H (2010) Functional Analysis, Sobolev Spaces and Partial Differential Equations (Springer-Verlag, New York).Google Scholar
• [22] Caffarelli L (1989) Interior a priori estimates for solutions of fully non-linear equations. Ann. Math. 130(1):189–213.Crossref, Google Scholar
• [23] Dean S, Mania H, Matni N, Recht B, Tu S (2020) On the sample complexity of the linear quadratic regulator. Foundations Comput. Math. 20:633–679.Crossref, Google Scholar
• [24] Dupuis P, James MR, Petersen I (2000) Robust properties of risk-sensitive control. Math. Control Signals Systems 13(4):318–332.Crossref, Google Scholar
• [25] Erdoğan E, Iyengar GN (2005) Ambiguous chance constrained problems and robust optimization. Math. Programming 107(1–2):37–61.Crossref, Google Scholar
• [26] Esfahani PM, Kuhn D (2018) Data-driven distributionally robust optimization using the Wasserstein metric: Performance guarantees and tractable reformulations. Math. Programming 171:115–166.Crossref, Google Scholar
• [27] Gilbarg D, Trudinger NS (1983) Elliptic Partial Differential Equations of Second Order, Grundlehren der Mathematischen Wissenschaften, vol. 224, 2nd ed. (Springer-Verlag, Berlin).Google Scholar
• [28] Gordienko E, Lemus-Rodríguez E (1999) Estimation of robustness for controlled diffusion processes. Stochastics Anal. Appl. 17(3):421–441.Crossref, Google Scholar
• [29] Gossner O, Tomala T (2008) Entropy bounds on Bayesian learning. J. Math. Econom. 44(1):24–32.Crossref, Google Scholar
• [30] Hansen LP, Sargent TJ (2001) Robust control and model uncertainty. Amer. Econom. Rev. 91(2):60–66.Crossref, Google Scholar
• [31] Ingram S (1972) Continuous dependence on parametem and boundary data for nonlinear two-point boundary value problems. Pacific J. Math. 41(2):395–408.Crossref, Google Scholar
• [32] Iyengar G (2005) Robust dynamic programming. Math. Oper. Res. 30(2):257–280.Link, Google Scholar
• [33] Jacobson D (1973) Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games. IEEE Trans. Automatic Control 18(2):124–131.Crossref, Google Scholar
• [34] Jiuying D (2007) Optimal investment consumption model with Vasicek interest rate. Chinese Control Conf. (IEEE, Piscataway, NJ), 391–394.Google Scholar
• [35] Kara AD, Yüksel S (2020) Robustness to incorrect system models in stochastic control. SIAM J. Control Optim. 58(2):1144–1182.Crossref, Google Scholar
• [36] Kara AD, Raginsky M, Yüksel S (2020) Robustness to incorrect models and data-driven learning in average-cost optimal stochastic control. Preprint, submitted March 11, https://arxiv.org/abs/2003.05769.Google Scholar
• [37] Kleptsyna ML, Veretennikov AY (2016) On robustness of discrete time optimal filters. Math. Methods Statist. 25(3):207–218.Crossref, Google Scholar
• [38] Knispel T (2012) Asymptotics of robust utility maximization. Ann. Appl. Probab. 22(1):172–212.Crossref, Google Scholar
• [39] Kushner HJ (1990) Weak Convergence Methods and Singularly Perturbed Control and Filtering Problems (Birkhäuser, Boston).Crossref, Google Scholar
• [40] Kushner HJ, Ramachandran KM (1988) Nearly optimal singular controls for wideband noise driven systems. SIAM J. Control Optim. 26(3):569–591.Crossref, Google Scholar
• [41] Kushner HJ, Runggaldier WJ (1987) Filtering and control for wide bandwidth noise driven systems. IEEE Trans. Automatic Control. 32(2):123–133.Crossref, Google Scholar
• [42] Kushner HJ, Runggaldier WJ (1987) Nearly optimal state feedback controls for stochastic systems with wideband noise disturbances. SIAM J. Control Optim. 25(2):298–315.Crossref, Google Scholar
• [43] Lam H (2016) Robust sensitivity analysis for stochastic systems. Math. Oper. Res. 41(4):1248–1275.Link, Google Scholar
• [44] Langen H (1981) Convergence of dynamic programming models. Math. Oper. Res. 6(4):493–512.Link, Google Scholar
• [45] Liptser R, Runggaldier WJ, Taksar M (2000) Diffusion approximation and optimal stochastic control. Theory Probab. Appl. 44(4):669–698.Crossref, Google Scholar
• [46] López-Barrientos J, Jasso-Fuentes H, Escobedo-Trujillo BA (2015) Discounted robust control for Markov diffusion processes. TOP 23:53–76.Crossref, Google Scholar
• [47] Merton RC (1998) Applications of option-pricing theory: Twenty-five years later. Amer. Econom. Rev. 88(3):323–349.Google Scholar
• [48] Meyn SP (1997) The policy iteration algorithm for average reward Markov decision processes with general state space. IEEE Trans. Automatic Control. 42(12):1663–1680.Crossref, Google Scholar
• [49] Müller A (1997) How does the value function of a Markov decision process depend on the transition probabilities? Math. Oper. Res. 22(4):872–885.Link, Google Scholar
• [50] Nilim A, Ghaoui LE (2005) Robust control of Markov decision processes with uncertain transition matrices. Oper. Res. 53(5):780–798.Link, Google Scholar
• [51] Øksendal B, Sulem A (2014) Forward–backward stochastic differential games and stochastic control under model uncertainty. J. Optim. Theory Appl. 161(1):22–55.Crossref, Google Scholar
• [52] Petersen I, James MR, Dupuis P (2000) Minimax optimal control of stochastic uncertain systems with relative entropy constraints. IEEE Trans. Automatic Control 45(3):398–412.Crossref, Google Scholar
• [53] Pham H (2009) Continuous-Time Stochastic Control and Applications with Financial Applications, Stochastic Modelling and Applied Probability, vol. 61 (Springer, Berlin Heidelberg).Google Scholar
• [54] Poznyak AS, Duncan T, Pasik-Duncan B, Boltyansky VG (2002) Robust maximum principle for multimodel lq-problem. Internat. J. Control 75(15):1170–1177.Crossref, Google Scholar
• [55] Poznyak AS, Duncan T, Pasik-Duncan B, Boltyansky VG (2002) Robust optimal control for minimax stochastic linear quadratic problem. Internat. J. Control 75(14):1054–1065.Crossref, Google Scholar
• [56] Poznyak AS, Duncan T, Pasik-Duncan B, Boltyansky VG (2002) Robust stochastic maximum principle for multimodel worst case optimization. Internat. J. Control 75(13):1032–1048.Crossref, Google Scholar
• [57] Pra PD, Meneghini L, Runggaldier WJ (1996) Connections between stochastic control and dynamic games. Math. Control Signals Systems 9(4):303–326.Crossref, Google Scholar
• [58] Pradhan S, Yüksel S (2022) Continuity of cost in Borkar control topology and implications on discrete space and time approximations for controlled diffusions under several criteria. Preprint, submitted September 29, https://arxiv.org/abs/2209.14982.Google Scholar
• [59] Reisinger C, Zhang Y (2021) Regularity and stability of feedback relaxed controls. SIAM J. Control Optim. 59(5):3118–3151.Crossref, Google Scholar
• [60] Saldi N, Linder T, Yüksel S (2018) Finite Approximations in Discrete-Time Stochastic Control: Quantized Models and Asymptotic Optimality (Springer, Cham, Switzerland).Crossref, Google Scholar
• [61] Saldi N, Yüksel S, Linder T (2017) On the asymptotic optimality of finite approximations to Markov decision processes with borel spaces. Math. Oper. Res. 42(4):945–978.Link, Google Scholar
• [62] Schied A (2008) Robust optimal control for a consumption-investment problem. Math. Methods Oper. Res. 67(1):1–20.Crossref, Google Scholar
• [63] Stroock DW, Varadhan SRS (1997) Multidimensional Diffusion Processes, vol. 233 (Springer Science & Business Media, Berlin Heidelberg).Google Scholar
• [64] Subramanian J, Sinha A, Mahajan A (2021) Robustness of Markov perfect equilibrium to model approximations in general-sum dynamic games. 2021 Seventh Indian Control Conf. (IEEE, Piscataway, NJ), 189–194.Google Scholar
• [65] Sun H, Xu H (2015) Convergence analysis for distributionally robust optimization and equilibrium problems. Math. Oper. Res. 41(2):377–401.Link, Google Scholar
• [66] Tzortzis I, Charalambous C, Charalambous T (2015) Dynamic programming subject to total variation distance ambiguity. SIAM J. Control Optim. 53(4):2040–2075.Crossref, Google Scholar
• [67] Walczak S, Ledzewicz U, Schättler H (2001) Stability of elliptic optimal control problems. Comput. Math. Appl. 41(10–11):1245–1256.Crossref, Google Scholar
• [68] Xu H, Mannor S (2010) Distributionally robust Markov decision processes. Lafferty JD, Williams CKI, Shawe-Taylor J, Zemel RS, Culotta A, eds. Adv. Neural Inform. Processing Systems, vol. 23 (Curran Associates, Inc.), 2505–2513.Google Scholar
• [69] Zhou K, Doyle JC, Glover K (1996) Robust and Optimal Control, vol. 40 (Prentice-Hall, Upper Saddle River, NJ).Google Scholar