A Game-Theoretic Framework for Generic Second-Order Traffic Flow Models Using Mean Field Games and Adversarial Inverse Reinforcement Learning

References

• Abbeel P, Ng AY (2004) Apprenticeship learning via inverse reinforcement learning. Proc. 21st Internat. Conf. Machine Learn. (Association for Computing Machinery, New York), 1–8.Google Scholar
• Achdou Y, Camilli F, Capuzzo-Dolcetta I (2012) Mean field games: Numerical methods for the planning problem. SIAM J. Control Optim. 50(1):77–109.Crossref, Google Scholar
• Achdou Y, Mannucci P, Marchi C, Tchou N (2020) Deterministic mean field games with control on the acceleration. Nonlinear Differential Equations Appl. NoDEA 27(3):1–32.Crossref, Google Scholar
• Achdou Y, Mannucci P, Marchi C, Tchou N (2021) Deterministic mean field games with control on the acceleration and state constraints. Preprint, submitted April 15, https://arxiv.org/abs/2104.07292.Google Scholar
• Albi G, Herty M, Kalise D, Segala C (2022) Moment-driven predictive control of mean-field collective dynamics. SIAM J. Control Optim. 60(2):814–841.Google Scholar
• Aw A, Rascle M (2000) Resurrection of “second order” models of traffic flow. SIAM J. Appl. Math. 60(3):916–938.Crossref, Google Scholar
• Aw A, Klar A, Rascle M, Materne T (2002) Derivation of continuum traffic flow models from microscopic follow-the-leader models. SIAM J. Appl. Math. 63(1):259–278.Crossref, Google Scholar
• Balzotti C, Iacomini E (2021) Stop-and-go waves: A microscopic and a macroscopic description. Mathematical Descriptions of Traffic Flow: Micro, Macro and Kinetic Models (Springer, Berlin), 63–78.Crossref, Google Scholar
• Benamou JD, Carlier G (2015) Augmented Lagrangian methods for transport optimization, mean field games and degenerate elliptic equations. J. Optim. Theory Appl. 167(1):1–26.Crossref, Google Scholar
• Cannarsa P, Capuani R, Cardaliaguet P (2021) Mean field games with state constraints: From mild to pointwise solutions of the PDE system. Calculation Variable Partial Differential Equations 60(3):108.Crossref, Google Scholar
• Capuani R, Marigonda A (2022) Constrained mean field games equilibria as fixed point of random lifting of set-valued maps. IFAC PapersOnLine 55(30):180–185.Crossref, Google Scholar
• Chen X, Liu S, Di X (2023a) A hybrid framework of reinforcement learning and physics-informed deep learning for spatiotemporal mean field games. Proc. Internat. Conf. Autonomous Agents Multiagent Systems (International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC), 1079–1087.Google Scholar
• Chen X, Liu S, Di X (2023b) Learning dual mean field games on graphs. Proc. 26th Eur. Conf. Artificial Intelligence (Kraków, Poland).Google Scholar
• Chevalier G, Le Ny J, Malhamé R (2015) A micro-macro traffic model based on mean-field games. Proc. Amer. Control Conf. (IEEE, Piscataway, NJ), 1983–1988.Google Scholar
• Chiarello FA, Piccoli B, Tosin A (2021) Multiscale control of generic second order traffic models by driver-assist vehicles. Multiscale Modeling Simulations 19(2):589–611.Crossref, Google Scholar
• Chow YT, Li W, Osher S, Yin W (2018) Algorithm for Hamilton-Jacobi equations in density space via a generalized Hopf formula. Preprint, submitted May 4, https://arxiv.org/abs/1805.01636.Google Scholar
• Coifman B, Li L (2017) A critical evaluation of the next generation simulation (NGSIM) vehicle trajectory data set. Transportation Res. Part B Methodological 105:362–377.Crossref, Google Scholar
• Costeseque G, Lebacque JP (2014) A variational formulation for higher order macroscopic traffic flow models: Numerical investigation. Transportation Res. Part B Methodological 70:112–133.Crossref, Google Scholar
• Costeseque G, Lebacque JP, Khelifi A (2015) Lagrangian GSOM traffic flow models on junctions. IFAC PapersOnLine 48(1):147–152.Google Scholar
• Couillet R, Perlaza SM, Tembine H, Debbah M (2012) Electrical vehicles in the smart grid: A mean field game analysis. IEEE J. Selected Areas Comm. 30(6):1086–1096.Crossref, Google Scholar
• Cristiani E, Iacomini E (2019) An interface-free multi-scale multi-order model for traffic flow. Discrete Continuous Dynamic Systems Ser. B 25(11):6189–6207.Google Scholar
• Cristiani E, Priuli FS (2014) A destination-preserving model for simulating wardrop equilibria in traffic flow on networks. Preprint, submitted September 1, https://arxiv.org/abs/1409.0350.Google Scholar
• Delle Monache ML, Piccoli B, Rossi F (2017) Traffic regulation via controlled speed limit. SIAM J. Control Optim. 55(5):2936–2958.Crossref, Google Scholar
• Di X, Shi R (2021) A survey on autonomous vehicle control in the era of mixed-autonomy: From physics-based to ai-guided driving policy learning. Transportation Res. Part C Emerging Tech. 125:103008.Crossref, Google Scholar
• Di X, Liu HX, Davis GA (2010) Hybrid extended Kalman filtering approach for traffic density estimation along signalized arterials: Use of global positioning system data. Transportation Res. Rec. 2188(1):165–173.Crossref, Google Scholar
• Di X, Shi R, Mo Z, Fu Y (2023) Physics-informed deep learning for traffic state estimation: A survey and the outlook. Algorithms (Basel) 16(6):305.Crossref, Google Scholar
• Djehiche B, Tcheukam A, Tembine H (2016) Mean-field-type games in engineering. Preprint, submitted May 11, https://arxiv.org/abs/1605.03281.Google Scholar
• Fan S, Herty M, Seibold B (2014) Comparative model accuracy of a data-fitted generalized Aw-Rascle-Zhang model. Networks Heterog. Media 9(2):239–268.Crossref, Google Scholar
• Festa A, Göttlich S (2017) A mean field games approach for multi-lane traffic management. Preprint, submitted November 11, https://arxiv.org/abs/1711.04116.Google Scholar
• Fiedler C, Herty M, Rom M, Segala C, Trimpe S (2023) Reproducing kernel Hilbert spaces in the mean field limit. Kinetic Related Models 16(6):850–870.Google Scholar
• Gong X, Piccoli B, Visconti G (2021) Mean-field of optimal control problems for hybrid model of multilane traffic. IEEE Control Systems Lett. 5(6):1964–1969.Crossref, Google Scholar
• Göttlich S, Iacomini E, Jung T (2020) Properties of the LWR model with time delay. Networks Heterog. Media 16(1):31–47.Crossref, Google Scholar
• Guéant O, Lasry JM, Lions PL (2011) Mean field games and applications. Paris-Princeton Lectures on Mathematical Finance 2010 (Springer, Berlin), 205–266.Crossref, Google Scholar
• Guo X, Hu A, Xu R, Zhang J (2019) Learning mean-field games. Adv. Neural Inform. Processing Systems (NeurIPS 2019), vol. 32 (Curran Associates, Inc., Red Hook, NY), 4963–4974.Google Scholar
• Huang M, Malhamé RP, Caines PE (2006) Large population stochastic dynamic games: Closed-loop Mckean-Vlasov systems and the Nash certainty equivalence principle. Comm. Inform. Systems 6(3):221–252.Crossref, Google Scholar
• Huang K, Chen X, Di X, Du Q (2021) Dynamic driving and routing games for autonomous vehicles on networks: A mean field game approach. Transportation Res. Part C Emerging Tech. 128:103189.Crossref, Google Scholar
• Huang K, Di X, Du Q, Chen X (2019) Stabilizing traffic via autonomous vehicles: A continuum mean field game approach. Proc. IEEE Intelligent Transportation Systems Conf. (IEEE, Piscataway, NJ), 3269–3274.Google Scholar
• Huang K, Di X, Du Q, Chen X (2020a) A game-theoretic framework for autonomous vehicles velocity control: Bridging microscopic differential games and macroscopic mean field games. Discrete Continuous Dynamic Systems Ser. B 25(12):4869–4903.Crossref, Google Scholar
• Huang K, Di X, Du Q, Chen X (2020b) Scalable traffic stability analysis in mixed-autonomy using continuum models. Transportation Res. Part C Emerging Tech. 111:616–630.Crossref, Google Scholar
• Kachroo P, Agarwal S, Sastry S (2016) Inverse problem for non-viscous mean field control: Example from traffic. IEEE Trans. Automated Control 61(11):3412–3421.Crossref, Google Scholar
• Khelifi A, Haj-Salem H, Lebacque JP, Nabli L (2016) Lagrangian discretization of generic second order models: Application to traffic control. Appl. Math. Inform. Sci. Internat. J. 10(4):1243–1254.Crossref, Google Scholar
• Lachapelle A, Wolfram MT (2011) On a mean field game approach modeling congestion and aversion in pedestrian crowds. Transportation Res. Part B Methodological 45(10):1572–1589.Crossref, Google Scholar
• Lachapelle A, Salomon J, Turinici G (2010) Computation of mean field equilibria in economics. Math. Models Methods Appl. Sci. 20(04):567–588.Crossref, Google Scholar
• Lasry JM, Lions PL (2007) Mean field games. Japanese J. Math. 2(1):229–260.Crossref, Google Scholar
• Lauriere M, Perrin S, Girgin S, Muller P, Jain A, Cabannes T, Piliouras G, et al. (2022) Scalable deep reinforcement learning algorithms for mean field games. Proc. 39th Internat. Conf. Machine Learn., Proceedings of Machine Learning Research, vol. 162 (PMLR, New York), 12078–12095.Google Scholar
• Lebacque JP, Khoshyaran MM (2013) A variational formulation for higher order macroscopic traffic flow models of the GSOM family. Proc. Soc. Behav. Sci. 80:370–394.Crossref, Google Scholar
• Lebacque JP, Mammar S, Salem HH (2007) Generic second order traffic flow modelling. Transportation and Traffic Theory 2007, 755–776.Google Scholar
• LeVeque RJ (2002) Finite Volume Methods for Hyperbolic Problems, vol. 31 (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• Li J, Zhang H (2013) The variational formulation of a non-equilibrium traffic flow model: Theory and implications. Proc. Soc. Behav. Sci. 80:327–340.Crossref, Google Scholar
• Lighthill MJ, Whitham GB (1955) On kinematic waves II. A theory of traffic flow on long crowded roads. Proc. Roy. Soc. London A Math. Phys. Sci. 229(1178):317–345.Crossref, Google Scholar
• Mo Z, Fu Y, Di X (2022a) Quantifying uncertainty in traffic state estimation using generative adversarial networks. Proc. IEEE 25th Internat. Conf. Intelligent Transportation Systems (IEEE, Piscataway, NJ), 2769–2774.Google Scholar
• Mo Z, Fu Y, Xu D, Di X (2022b) Trafficflowgan: Physics-informed flow based generative adversarial network for uncertainty quantification. Proc. Joint Eur. Conf. Machine Learn. Knowledge Discovery Databases (Springer, Berlin), 323–339.Google Scholar
• Nisio M (2015) Viscosity Solutions for HJB Equations (Springer Japan, Tokyo).Crossref, Google Scholar
• Perrin S, Perolat J, Laurière M, Geist M, Elie R, Pietquin O (2020) Fictitious play for mean field games: Continuous time analysis and applications. Proc. 34th Internat. Conf. Neural Inform. Processing Systems (ACM, New York).Google Scholar
• Richards PI (1956) Shock waves on the highway. Oper. Res. 4(1):42–51.Link, Google Scholar
• Ruan K, Di X (2022) Learning human driving behaviors with sequential causal imitation learning. Proc. Conf. AAAI Artificial Intelligence 36:4583–4592.Crossref, Google Scholar
• Ruan K, Zhang J, Di X, Bareinboim E (2023) Causal imitation learning via inverse reinforcement learning. Proc. 11th Internat. Conf. Learn. Representations (ICLR, Appleton, WI).Google Scholar
• Seibold B, Flynn MR, Kasimov AR, Rosales RR (2013) Constructing set-valued fundamental diagrams from Jamiton solutions in second order traffic models. Networks Heterog. Media 8:745–772.Crossref, Google Scholar
• Shi R, Mo Z, Di X (2021a) Physics-informed deep learning for traffic state estimation: A hybrid paradigm informed by second-order traffic models. Proc. Conf. AAAI Artificial Intelligence 35:540–547.Crossref, Google Scholar
• Shi R, Mo Z, Huang K, Di X, Du Q (2021b) A physics-informed deep learning paradigm for traffic state and fundamental diagram estimation. IEEE Trans. Intelligent Transportation Systems 23(8):11688–11698.Crossref, Google Scholar
• Shou Z, Chen X, Fu Y, Di X (2022) Multi-agent reinforcement learning for Markov routing games: A new modeling paradigm for dynamic traffic assignment. Transportation Res. Part C Emerging Tech. 137:103560.Crossref, Google Scholar
• Syed U, Schapire RE (2007) A game-theoretic approach to apprenticeship learning. Proc. 20th Internat. Conf. Neural Inform. Processing Systems (Curran Associates, Inc., Red Hook, NY), 1449–1456.Google Scholar
• Wang Y, Papageorgiou M (2005) Real-time freeway traffic state estimation based on extended Kalman filter: A general approach. Transportation Res. Part B Methodological 39(2):141–167.Crossref, Google Scholar
• Yu H, Bayen AM, Krstic M (2019) Boundary observer for congested freeway traffic state estimation via Aw-Rascle-Zhang model. IFAC PapersOnLine 52(2):183–188.Google Scholar
• Zhang HM (2002) A non-equilibrium traffic model devoid of gas-like behavior. Transportation Res. Part B Methodological 36(3):275–290.Crossref, Google Scholar
• Zhou F, Zhang C, Chen X, Di X (2024) Graphon mean field games with a representative player: Analysis and learning algorithm. Preprint, submitted May 8, https://arxiv.org/abs/2405.08005.Google Scholar