Robustness and Approximation of Discrete-Time Mean-Field Games Under Discounted Cost Criterion

References

• [1] Achdou Y, Capuzzo-Dolcetta I (2010) Mean field games: Numerical methods. SIAM J. Numer. Anal. 48(3):1136–1162.Crossref, Google Scholar
• [2] Achdou Y, Camilli F, Capuzzo-Dolcetta I (2012) Mean field games: Numerical methods for the planning problem. SIAM J. Control Optim. 50(1):77–109.Crossref, Google Scholar
• [3] Adlakha S, Johari R, Weintraub GY (2015) Equilibria of dynamic games with many players: Existence, approximation, and market structure. J. Econom. Theory 156:269–316.Crossref, Google Scholar
• [4] Almulla N, Ferreira R, Gomes D (2017) Two numerical approaches to stationary mean-field games. Dynam. Games Appl. 7(4):657–682.Crossref, Google Scholar
• [5] Anahtarcı B, Karıksız CD, Saldi N (2020) Value iteration algorithm for mean-field games. Systems Control Lett. 143:104744.Crossref, Google Scholar
• [6] Anahtarci B, Kariksiz CD, Saldi N (2023) Learning mean-field games with discounted and average costs. J. Machine Learn. Res. 24(17):1–59.Google Scholar
• [7] Anahtarci B, Kariksiz CD, Saldi N (2023) Q-learning in regularized mean-field games. Dynam. Games Appl. 13(1):89–117.Google Scholar
• [8] Baker G, Yüksel S (2016) Continuity and robustness to incorrect priors in estimation and control. Guillén i Fàbregas A, Martínez A, Verdú S, eds. 2016 IEEE Internat. Sympos. Inform. Theory (ISIT) (IEEE, Piscataway, NJ), 1999–2003.Google Scholar
• [9] Bauso D, Tembine H, Başar T (2016) Robust mean field games. Dynam. Games Appl. 6(3):277–303.Crossref, Google Scholar
• [10] Bensoussan A, Frehse J, Yam P (2013) Mean Field Games and Mean Field Type Control Theory. SpringerBriefs in Mathematics, vol. 101 (Springer, New York).Crossref, Google Scholar
• [11] Bertsekas DP, Shreve SE (1978) Stochastic Optimal Control. Mathematics in Science and Engineering, vol. 139 (Academic Press, Inc., New York).Google Scholar
• [12] Billingsley P (1999) Convergence of Probability Measures. Wiley Series in Probability and Statistics: Probability and Statistics, 2nd ed. (John Wiley & Sons, Inc., New York).Crossref, Google Scholar
• [13] Biswas A (2015) Mean field games with ergodic cost for discrete time Markov processes. Preprint, submitted October 30, https://arxiv.org/abs/1510.08968.Google Scholar
• [14] Bogachev VI (2007) Measure Theory, vol. I, II (Springer-Verlag, Berlin).Crossref, Google Scholar
• [15] Braides A (2002) Γ-Convergence for Beginners. Oxford Lecture Series in Mathematics and Its Applications, vol. 22 (Oxford University Press, Oxford, UK).Google Scholar
• [16] Cardaliaguet P (2011) Notes on mean-field games. Working paper, Université Paris-Dauphine, Paris.Google Scholar
• [17] Cardaliaguet P, Delarue F, Lasry JM, Lions PL (2019) The Master Equation and the Convergence Problem in Mean Field Games (Princeton University Press, Princeton, NJ).Google Scholar
• [18] Carmona R, Delarue F (2013) Probabilistic analysis of mean-field games. SIAM J. Control Optim. 51(4):2705–2734.Crossref, Google Scholar
• [19] Cui K, Koeppl H (2021) Approximately solving mean field games via entropy-regularized deep reinforcement learning. Arindam B, Kenji F, eds. Proc. 24th Internat. Conf. Artificial Intelligence Statist., vol. 130 (PMLR, New York), 1909–1917.Google Scholar
• [20] Elliott R, Li X, Ni YH (2013) Discrete time mean-field stochastic linear-quadratic optimal control problems. Automatica 49(11):3222–3233.Crossref, Google Scholar
• [21] Gomes DA, Saúde J (2014) Mean field games models—A brief survey. Dynam. Games Appl. 4(2):110–154.Crossref, Google Scholar
• [22] Gomes DA, Mohr J, Souza RR (2011) Discrete time, finite state space mean field games. Peixoto M, Pinto A, Rand D, eds. Dynamics, Games and Science I. Springer Proceedings in Mathematics, vol. 1 (Springer, Berlin), 385–389.Crossref, Google Scholar
• [23] Guo X, Hu A, Xu R, Zhang J (2023) A general framework for learning mean-field games. Math. Oper. Res. 48(2):656–686.Link, Google Scholar
• [24] Hernández-Lerma O, Lasserre JB (2012) Discrete-Time Markov Control Processes: Basic Optimality Criteria. Stochastic Modelling and Applied Probability, vol. 30 (Springer Science & Business Media, New York).Google Scholar
• [25] Huang M (2010) Large-population LQG games involving a major player: The Nash certainty equivalence principle. SIAM J. Control Optim. 48(5):3318–3353.Crossref, Google Scholar
• [26] Huang M, Ma Y (2019) Binary mean field stochastic games: Stationary equilibria and comparative statics. Yin G, Zhang Q, eds. Modeling, Stochastic Control, Optimization, and Applications. The IMA Volumes in Mathematics and Its Applications, vol. 164 (Springer, Cham, Switzerland), 283–313.Crossref, Google Scholar
• [27] Huang M, Caines PE, Malhame RP (2007) Large-population cost-coupled LQG problems with nonuniform agents: Individual-mass behavior and decentralized ϵ-Nash equilibria. IEEE Trans. Automatic Control 52(9):1560–1571.Crossref, Google Scholar
• [28] Huang M, Malhamé RP, Caines PE (2006) Large population stochastic dynamic games: Closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle. Comm. Inform. Systems 6(3):221–252.Crossref, Google Scholar
• [29] Jusup M, Pásztor B, Janik T, Zhang K, Corman F, Krause A, Bogunovic I (2023) Safe model-based multi-agent mean-field reinforcement learning. Preprint, submitted June 29, https://arxiv.org/abs/2306.17052.Google Scholar
• [30] Kara AD, Yüksel S (2019) Robustness to incorrect priors in partially observed stochastic control. SIAM J. Control Optim. 57(3):1929–1964.Crossref, Google Scholar
• [31] Langen HJ (1981) Convergence of dynamic programming models. Math. Oper. Res. 6(4):493–512.Link, Google Scholar
• [32] Lasry JM, Lions PL (2007) Mean field games. Jpn. J. Math. 2(1):229–260.Crossref, Google Scholar
• [33] Laurière M (2021) Numerical methods for mean field games and mean field type control. Proc. Sympos. Appl. Math. 78:221–282.Crossref, Google Scholar
• [34] Milgrom PR, Weber RJ (1985) Distributional strategies for games with incomplete information. Math. Oper. Res. 10(4):619–632.Link, Google Scholar
• [35] Moon J, Başar T (2015) Discrete-time decentralized control using the risk-sensitive performance criterion in the large population regime: A mean field approach. Astolfi A, ed. 2015 Amer. Control Conf. (ACC) (IEEE, Piscataway, NJ), 4779–4784.Google Scholar
• [36] Moon J, Başar T (2016) Discrete-time mean field Stackelberg games with a large number of followers. Proc. 55th IEEE Conf. Decision Control (CDC ‘16) (IEEE, Piscataway, NJ), 3578–3583.Google Scholar
• [37] Moon J, Başar T (2016) Robust mean field games for coupled Markov jump linear systems. Internat. J. Control 89(7):1367–1381.Crossref, Google Scholar
• [38] Moon J, Başar T (2017) Linear quadratic risk-sensitive and robust mean field games. IEEE Trans. Automatic Control 62(3):1062–1077.Crossref, Google Scholar
• [39] Nilim A, El Ghaoui L (2005) Robust control of Markov decision processes with uncertain transition matrices. Oper. Res. 53(5):780–798.Link, Google Scholar
• [40] Nourian M, Nair GN (2013) Linear-quadratic-Gaussian mean field games under high rate quantization. 52nd IEEE Conf. Decision Control (IEEE, Piscataway, NJ), 1898–1903.Google Scholar
• [41] Pásztor B, Krause A, Bogunovic I (2023) Efficient model-based multi-agent mean-field reinforcement learning. Trans. Machine Learn. Res. (OpenReview.net).Google Scholar
• [42] Saldi N (2020) Discrete-time average-cost mean-field games on Polish spaces. Turkish J. Math. 44(2):463–480.Google Scholar
• [43] Saldi N, Başar T, Raginsky M (2018) Markov–Nash equilibria in mean-field games with discounted cost. SIAM J. Control Optim. 56(6):4256–4287.Crossref, Google Scholar
• [44] Saldi N, Başar T, Raginsky M (2019) Approximate Nash equilibria in partially observed stochastic games with mean-field interactions. Math. Oper. Res. 44(3):1006–1033.Link, Google Scholar
• [45] Saldi N, Başar T, Raginsky M (2020) Approximate Markov-Nash equilibria for discrete-time risk-sensitive mean-field games. Math. Oper. Res. 45(4):1596–1620.Link, Google Scholar
• [46] Saldi N, Başar T, Raginsky M (2023) Partially observed discrete-time risk-sensitive mean field games. Dynam. Games Appl. 13(3):929–960.Crossref, Google Scholar
• [47] Saldi N, Yüksel S, Linder T (2017) On the asymptotic optimality of finite approximations to Markov decision processes with Borel spaces. Math. Oper. Res. 42(4):945–978.Link, Google Scholar
• [48] Satia JK, Lave RE Jr (1973) Markovian decision processes with uncertain transition probabilities. Oper. Res. 21(3):728–740.Link, Google Scholar
• [49] Serfozo R (1982) Convergence of Lebesgue integrals with varying measures. Sankhyā Ser. A 44(3):380–402.Google Scholar
• [50] Shreve SE, Bertsekas DP (1979) Universally measurable policies in dynamic programming. Math. Oper. Res. 4(1):15–30.Link, Google Scholar
• [51] Subramanian J, Mahajan A (2019) Reinforcement learning in stationary mean-field games. Elkind E, Veloso M, Agmon N, Taylor ME, eds. Proc. 18th Internat. Conf. Autonomous Agents MultiAgent Systems (International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC), 251–259.Google Scholar
• [52] Tembine H, Zhu Q, Başar T (2014) Risk-sensitive mean-field games. IEEE Trans. Automatic Control 59(4):835–850.Crossref, Google Scholar
• [53] Villani C (2009) Optimal Transport: Old and New. Grundlehren der mathematischen Wissenschaften, vol. 338 (Springer, Berlin).Crossref, Google Scholar
• [54] Weintraub GY, Benkard L, Van Roy B (2005) Oblivious equilibrium: A mean field approximation for large-scale dynamic games. Weiss Y, Schölkopf B, Platt J, eds. Adv. Neural Inform. Processing Systems 18 (NIPS 2005), 1489–1496.Google Scholar
• [55] Weintraub GY, Benkard CL, Van Roy B (2008) Markov perfect industry dynamics with many firms. Econometrica 76(6):1375–1411.Crossref, Google Scholar
• [56] Więcek P (2020) Discrete-time ergodic mean-field games with average reward on compact spaces. Dynam. Games Appl. 10(1):222–256.Crossref, Google Scholar
• [57] Więcek P, Altman E (2015) Stationary anonymous sequential games with undiscounted rewards. J. Optim. Theory Appl. 166(2):686–710.Crossref, Google Scholar
• [58] Zaman MAU, Koppel A, Bhatt S, Başar T (2023) Oracle-free reinforcement learning in mean-field games along a single sample path. Ruiz F, Dy J, van de Meent J-W, eds. Internat. Conf. Artificial Intelligence Statist., vol. 206 (PMLR, New York), 10178–10206.Google Scholar