An Iterative Adaptive Dynamic Programming Approach for Macroscopic Fundamental Diagram-Based Perimeter Control and Route Guidance

References

• Aboudolas K, Geroliminis N (2013) Perimeter and boundary flow control in multi-reservoir heterogeneous networks. Transportation Res. Part B Methodological 55:265–281.Crossref, Google Scholar
• Ambühl L, Menendez M (2016) Data fusion algorithm for macroscopic fundamental diagram estimation. Transportation Res. Part C Emerging Tech. 71:184–197.Crossref, Google Scholar
• Ambühl L, Loder A, Bliemer MC, Menendez M, Axhausen KW (2018) Introducing a re-sampling methodology for the estimation of empirical macroscopic fundamental diagrams. Transportation Res. Rec. 2672(20):239–248.Crossref, Google Scholar
• Ameli M, Faradonbeh MSS, Lebacque JP, Abouee-Mehrizi H, Leclercq L (2022) Departure time choice models in urban transportation systems based on mean field games. Transport. Sci. 56(6):1483–1504.Link, Google Scholar
• Ampountolas K, Zheng N, Geroliminis N (2017) Macroscopic modelling and robust control of bi-modal multi-region urban road networks. Transportation Res. Part B Methodological 104:616–637.Crossref, Google Scholar
• Baldi S, Michailidis I, Ntampasi V, Kosmatopoulos E, Papamichail I, Papageorgiou M (2019) A simulation-based traffic signal control for congested urban traffic networks. Transportation Sci. 53(1):6–20.Link, Google Scholar
• Batista SF, Leclercq L (2019) Regional dynamic traffic assignment framework for macroscopic fundamental diagram multi-regions models. Transportation Sci. 53(6):1563–1590.Link, Google Scholar
• Batista S, Leclercq L, Geroliminis N (2019) Estimation of regional trip length distributions for the calibration of the aggregated network traffic models. Transportation Res. Part B Methodological 122:192–217.Crossref, Google Scholar
• Batista SF, Ingole D, Leclercq L, Menéndez M (2021) The role of trip lengths calibration in model-based perimeter control strategies. IEEE Trans. Intelligent Transportation Systems 23(6):5176–5186.Crossref, Google Scholar
• Buisson C, Ladier C (2009) Exploring the impact of homogeneity of traffic measurements on the existence of macroscopic fundamental diagrams. Transportation Res. Rec. 2124(1):127–136.Crossref, Google Scholar
• Cao J, Menendez M (2015) System dynamics of urban traffic based on its parking-related-states. Transportation Res. Part B Methodological 81:718–736.Crossref, Google Scholar
• Chen C, Huang Y, Lam W, Pan T, Hsu S, Sumalee A, Zhong R (2022) Data efficient reinforcement learning and adaptive optimal perimeter control of network traffic dynamics. Transportation Res. Part C Emerging Tech. 142:103759.Crossref, Google Scholar
• Daganzo CF (2007) Urban gridlock: Macroscopic modeling and mitigation approaches. Transportation Res. Part B Methodological 41(1):49–62.Crossref, Google Scholar
• Ding H, Di Y, Feng Z, Zhang W, Zheng X, Yang T (2022) A perimeter control method for a congested urban road network with dynamic and variable ranges. Transportation Res. Part B Methodological 155:160–187.Crossref, Google Scholar
• Fu H, Liu N, Hu G (2017) Hierarchical perimeter control with guaranteed stability for dynamically coupled heterogeneous urban traffic. Transportation Res. Part C Emerging. Tech. 83:18–38.Crossref, Google Scholar
• Fu H, Chen S, Chen K, Kouvelas A, Geroliminis N (2021) Perimeter control and route guidance of multi-region MFD systems with boundary queues using colored petri nets. IEEE Trans. Intelligent Transportation Systems 23(8):12977–12999.Google Scholar
• Gao S, Li D, Zheng N, Hu R, She Z (2022) Resilient perimeter control for hyper-congested two-region networks with MFD dynamics. Transportation Res. Part B Methodological 156:50–75.Crossref, Google Scholar
• Geroliminis N, Boyaci B (2012) The effect of variability of urban systems characteristics in the network capacity. Transportation Res. Part B Methodological 46(10):1607–1623.Crossref, Google Scholar
• Geroliminis N, Daganzo CF (2008) Existence of urban-scale macroscopic fundamental diagrams: Some experimental findings. Transportation Res. Part B Methodological 42(9):759–770.Crossref, Google Scholar
• Geroliminis N, Sun J (2011) Properties of a well-defined macroscopic fundamental diagram for urban traffic. Transportation Res. Part B Methodological 45(3):605–617.Crossref, Google Scholar
• Geroliminis N, Haddad J, Ramezani M (2013) Optimal perimeter control for two urban regions with macroscopic fundamental diagrams: A model predictive approach. IEEE Trans. Intelligent Transportation Systems 14(1):348–359.Crossref, Google Scholar
• Godfrey J (1969) The mechanism of a road network. Traffic Engrg. Control 11:323–327.Google Scholar
• Gu Z, Shafiei S, Liu Z, Saberi M (2018) Optimal distance-and time-dependent area-based pricing with the network fundamental diagram. Transporation Res. Part C Emerging Tech. 95:1–28.Crossref, Google Scholar
• Haddad J (2015) Robust constrained control of uncertain macroscopic fundamental diagram networks. Transportation Res. Part C Emerging Tech. 59:323–339.Crossref, Google Scholar
• Haddad J, Mirkin B (2020) Resilient perimeter control of macroscopic fundamental diagram networks under cyberattacks. Transportation Res. Part B Methodological 132:44–59.Crossref, Google Scholar
• Haddad J, Shraiber A (2014) Robust perimeter control design for an urban region. Transportation Res. Part B Methodological 68:315–332.Crossref, Google Scholar
• Haddad J, Ramezani M, Geroliminis N (2013) Cooperative traffic control of a mixed network with two urban regions and a freeway. Transportation Res. Part B Methodological 54:17–36.Crossref, Google Scholar
• Hamedmoghadam H, Zheng N, Li D, Vu HL (2022) Percolation-based dynamic perimeter control for mitigating congestion propagation in urban road networks. Transportation Res. Part C Emerging Tech. 145:103922.Crossref, Google Scholar
• Hou Z, Lei T (2020) Constrained model free adaptive predictive perimeter control and route guidance for multi-region urban traffic systems. IEEE Trans. Intelligent Transportation Systems 23(2):912–924.Crossref, Google Scholar
• Hu Z, Ma W (2024) Demonstration-guided deep reinforcement learning for coordinated ramp metering and perimeter control in large scale networks. Transportation Res. Part C Emerging Tech. 159:104461.Crossref, Google Scholar
• Huang Y, Xiong J, Sumalee A, Zheng N, Lam W, He Z, Zhong R (2020) A dynamic user equilibrium model for multi-region macroscopic fundamental diagram systems with time-varying delays. Transportation Res. Part B Methodological 131:1–25.Crossref, Google Scholar
• Jiang S, Tran CQ, Keyvan-Ekbatani M (2024) Regional route guidance with realistic compliance patterns: Application of deep reinforcement learning and MPC. Transportation Res. Part C Emerging Tech. 158:104440.Crossref, Google Scholar
• Keyvan-Ekbatani M, Papageorgiou M, Knoop VL (2015) Controller design for gating traffic control in presence of time-delay in urban road networks. Transportation Res. Proc. 7:651–668.Crossref, Google Scholar
• Keyvan-Ekbatani M, Kouvelas A, Papamichail I, Papageorgiou M (2012) Exploiting the fundamental diagram of urban networks for feedback-based gating. Transportation Res. Part B Methodological 46(10):1393–1403.Crossref, Google Scholar
• Knoop V, Hoogendoorn S, Van Lint J (2012) Routing strategies based on macroscopic fundamental diagram. Transportation Res. Rec. 2315(1):1–10.Crossref, Google Scholar
• Kouvelas A, Saeedmanesh M, Geroliminis N (2017) Enhancing model-based feedback perimeter control with data-driven online adaptive optimization. Transportation Res. Part B Methodological 96:26–45.Crossref, Google Scholar
• Kouvelas A, Saeedmanesh M, Geroliminis N (2023) A linear-parameter-varying formulation for model predictive perimeter control in multi-region MFD urban networks. Transportation Sci. 57(6):1496–1515.Link, Google Scholar
• Kumarage S, Yildirimoglu M, Ramezani M, Zheng Z (2021) Schedule-constrained demand management in two-region urban networks. Transportation Sci. 55(4):857–882.Link, Google Scholar
• Leclercq L, Geroliminis N (2013) Estimating MFDs in simple networks with route choice. Transportation Res. Part B Methodological 57:468–484.Crossref, Google Scholar
• Leclercq L, Sénécat A, Mariotte G (2017) Dynamic macroscopic simulation of on-street parking search: A trip-based approach. Transportation Res. Part B Methodological 101:268–282.Crossref, Google Scholar
• Lee J, Sutton RS (2021) Policy iterations for reinforcement learning problems in continuous time and space: Fundamental theory and methods. Automatica J. IFAC 126:109421.Crossref, Google Scholar
• Lei T, Hou Z, Ren Y (2019) Data-driven model free adaptive perimeter control for multi-region urban traffic networks with route choice. IEEE Trans. Intelligent Transportation Systems 21(7):2894–2905.Google Scholar
• Li Y, Mohajerpoor R, Ramezani M (2021) Perimeter control with real-time location-varying cordon. Transportation Res. Part B Methodological 150:101–120.Crossref, Google Scholar
• Mahmassani HS, Williams JC, Herman R (1984) Investigation of network-level traffic flow relationships: Some simulation results. Transportation Res. Rec. 971:121–130.Google Scholar
• Mariotte G, Leclercq L, Batista S, Krug J, Paipuri M (2020) Calibration and validation of multi-reservoir MFD models: A case study in Lyon. Transportation Res. Part B Methodological 136:62–86.Crossref, Google Scholar
• Mercader P, Haddad J (2021) Resilient multivariable perimeter control of urban road networks under cyberattacks. Control Engrg. Practice 109:104718.Crossref, Google Scholar
• Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, et al. (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529.Crossref, Google Scholar
• Modares H, Lewis FL, Naghibi-Sistani MB (2014) Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems. Automatica J. IFAC 50:193–202.Crossref, Google Scholar
• Mohajerpoor R, Saberi M, Vu HL, Garoni TM, Ramezani M (2020) H ∞ robust perimeter flow control in urban networks with partial information feedback. Transportation Res. Part B Methodological 137:47–73.Crossref, Google Scholar
• Moshahedi N, Kattan L (2023) Alpha-fair large-scale urban network control: A perimeter control based on a macroscopic fundamental diagram. Transportation Res. Part C Emerging Tech. 146:103961.Crossref, Google Scholar
• Murray JJ, Cox CJ, Lendaris GG, Saeks R (2002) Adaptive dynamic programming. IEEE Trans. Systems Man Cybernics C 32(2):140–153.Crossref, Google Scholar
• Ogata K (2010) Modern Control Engineering, vol. 5 (Prentice Hall, Upper Saddle River, NJ).Google Scholar
• Paipuri M, Barmpounakis E, Geroliminis N, Leclercq L (2021) Empirical observations of multi-modal network-level models: Insights from the pneuma experiment. Transportation Res. Part C Emerging Tech. 131:103300.Crossref, Google Scholar
• Ramezani M, Valadkhani AH (2023) Dynamic ride-sourcing systems for city-scale networks-part I: Matching design and model formulation and validation. Transportation Res. Part C Emerging Tech. 152:104158.Crossref, Google Scholar
• Ramezani M, Haddad J, Geroliminis N (2015) Dynamics of heterogeneity in urban networks: Aggregated traffic modeling and hierarchical control. Transportation Res. Part B Methodological 74:1–19.Crossref, Google Scholar
• Ren Y, Hou Z, Sirmatel II, Geroliminis N (2020) Data driven model free adaptive iterative learning perimeter control for large-scale urban road networks. Transportation Res. Part C Emerging Tech. 115:102618.Crossref, Google Scholar
• Saberi M, Mahmassani HS (2013) Hysteresis and capacity drop phenomena in freeway networks: Empirical characterization and interpretation. Transportation Res. Rec. 2391(1):44–55.Crossref, Google Scholar
• Saeedmanesh M, Geroliminis N (2016) Clustering of heterogeneous networks with directional flows based on “snake” similarities. Transportation Res. Part B Methodological 91:250–269.Crossref, Google Scholar
• Saeedmanesh M, Geroliminis N (2017) Dynamic clustering and propagation of congestion in heterogeneously congested urban traffic networks. Transportation Res. Proc. 23:962–979.Crossref, Google Scholar
• Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, et al. (2016) Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484.Crossref, Google Scholar
• Sirmatel II, Geroliminis N (2018) Economic model predictive control of large-scale urban road networks via perimeter control and regional route guidance. IEEE Trans. Intelligent Transportation Systems 19(4):1112–1121.Crossref, Google Scholar
• Sirmatel II, Geroliminis N (2019) Nonlinear moving horizon estimation for large-scale urban road networks. IEEE Trans. Intelligent Transportation Systems 21(12):4983–4994.Crossref, Google Scholar
• Sirmatel II, Tsitsokas D, Kouvelas A, Geroliminis N (2021) Modeling, estimation, and control in large-scale urban road networks with remaining travel distance dynamics. Transportation Res. Part C Emerging Tech. 128:103157.Crossref, Google Scholar
• Su Z, Chow AH, Fang C, Liang E, Zhong R (2023) Hierarchical control for stochastic network traffic with reinforcement learning. Transportation Res. Part B Methodological 167:196–216.Crossref, Google Scholar
• Su Z, Chow AH, Zheng N, Huang Y, Liang E, Zhong R (2020) Neuro-dynamic programming for optimal control of macroscopic fundamental diagram systems. Transportation Res. Part C Emerging Tech. 116:102628.Crossref, Google Scholar
• Sutton RS, Barto AG (2018) Reinforcement Learning: An Introduction (MIT Press, Cambridge, MA).Google Scholar
• Tsitsokas D, Kouvelas A, Geroliminis N (2023) Two-layer adaptive signal control framework for large-scale dynamically-congested networks: Combining efficient max pressure with perimeter control. Transportation Res. Part C Emerging Tech. 152:104128.Crossref, Google Scholar
• Tsubota T, Bhaskar A, Chung E (2014) Macroscopic fundamental diagram for Brisbane, Australia: Empirical findings on network partitioning and incident detection. Transportation Res. Rec. 2421(1):12–21.Crossref, Google Scholar
• Valadkhani AH, Ramezani M (2023) Dynamic ride-sourcing systems for city-scale networks, part II: Proactive vehicle repositioning. Transportation Res. Part C Emerging Tech. 152:104159.Crossref, Google Scholar
• Wei B, Saberi M, Zhang F, Liu W, Waller ST (2020) Modeling and managing ridesharing in a multi-modal network with an aggregate traffic representation: A doubly dynamical approach. Transportation Res. Part C Emerging Tech. 117:102670.Crossref, Google Scholar
• Yildirimoglu M, Geroliminis N (2014) Approximating dynamic equilibrium conditions with macroscopic fundamental diagrams. Transportation Res. Part B Methodological 70:186–200.Crossref, Google Scholar
• Yildirimoglu M, Ramezani M (2020) Demand management with limited cooperation among travellers: A doubly dynamic approach. Transportation Res. Part B Methodological 132:267–284.Crossref, Google Scholar
• Yildirimoglu M, Ramezani M, Geroliminis N (2015) Equilibrium analysis and route guidance in large-scale networks with MFD dynamics. Transportation Res. Proc. 9:185–204.Crossref, Google Scholar
• Yildirimoglu M, Sirmatel II, Geroliminis N (2018) Hierarchical control of heterogeneous large-scale urban road networks via path assignment and regional route guidance. Transportation Res. Part B Methodological 118:106–123.Crossref, Google Scholar
• Zheng N, Geroliminis N (2020) Area-based equitable pricing strategies for multimodal urban networks with heterogeneous users. Transportation Res. Part A Policy Practice 136:357–374.Crossref, Google Scholar
• Zhong R, Sumalee A, Pan T, Lam W (2014) Optimal and robust strategies for freeway traffic management under demand and supply uncertainties: An overview and general theory. Transportmetrica A Transportation Sci. 10(10):849–877.Crossref, Google Scholar
• Zhong R, Chen C, Huang Y, Sumalee A, Lam W, Xu D (2018a) Robust perimeter control for two urban regions with macroscopic fundamental diagrams: A control-lyapunov function approach. Transportation Res. Part B Methodological 117:687–707.Crossref, Google Scholar
• Zhong R, Huang Y, Chen C, Lam W, Xu D, Sumalee A (2018b) Boundary conditions and behavior of the macroscopic fundamental diagram based network traffic dynamics: A control systems perspective. Transportation Res. Part B Methodological 111:327–355.Crossref, Google Scholar
• Zhong R, Xiong J, Huang Y, Sumalee A, Chow AH, Pan T (2020) Dynamic system optimum analysis of multi-region macroscopic fundamental diagram systems with state-dependent time-varying delays. IEEE Trans. Intelligent Transportation Systems 21:4000–4016.Crossref, Google Scholar
• Zhong R, Xiong J, Huang Y, Zheng N, Lam WH, Pan T, He B (2021) Dynamic user equilibrium for departure time choice in the basic trip-based model. Transportation Res. Part C Emerging Tech. 128:103190.Crossref, Google Scholar
• Zhou D, Gayah VV (2021) Model-free perimeter metering control for two-region urban networks using deep reinforcement learning. Transportation Res. Part C Emerging Tech. 124:102949.Crossref, Google Scholar
• Zhou Z, De Schutter B, Lin S, Xi Y (2016) Two-level hierarchical model-based predictive control for large-scale urban traffic networks. IEEE Trans. Control Systems Tech. 25(2):496–508.Crossref, Google Scholar