Cooperative Learning for Smart Charging of Shared Autonomous Vehicle Fleets

References

• Abdelwahed A, van den Berg PL, Brandt T, Collins J, Ketter W (2020) Evaluating and optimizing opportunity fast-charging schedules in transit battery electric bus networks. Transportation Sci. 54(6):1601–1615.Link, Google Scholar
• Abouee-Mehrizi H, Baron O, Berman O, Chen D (2021) Adoption of electric vehicles in car sharing market. Production Oper. Management 30(1):190–209.Google Scholar
• Ahadi R, Ketter W, Collins J, Daina N (2021) Siting and sizing of charging infrastructure for shared autonomous electric fleets. Endriss U, Nowé A, Dignum F, Lomuscio A, eds. Proc. 20th Internat. Conf. Autonomous Agents MultiAgent Systems (International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC), 88–96.Google Scholar
• Al-Kanj L, Nascimento J, Powell WB (2020) Approximate dynamic programming for planning a ride-hailing system using autonomous fleets of electric vehicles. Eur. J. Operational Res. 284(3):1088–1106.Crossref, Google Scholar
• Bösch PM, Becker F, Becker H, Axhausen KW (2018) Cost-based analysis of autonomous mobility services. Transportation Policy 64:76–91.Crossref, Google Scholar
• Chen Y, Liu Y (2022) Integrated optimization of planning and operations for shared autonomous electric vehicle systems. Transportation Sci., ePub ahead of print August 24, https://doi.org/10.1287/trsc.2022.1156.Google Scholar
• Chen TD, Kockelman KM, Hanna JP (2016) Operations of a shared, autonomous, electric vehicle fleet: Implications of vehicle & charging infrastructure decisions. Transportation Res. Part A: Policy Practice 94:243–254.Crossref, Google Scholar
• Daina N, Sivakumar A, Polak JW (2017) Electric vehicle charging choices: Modelling and implications for smart charging services. Transportation Res. Part C: Emerging Tech. 81:36–56.Crossref, Google Scholar
• Devlin S, Kudenko D (2011) Theoretical considerations of potential-based reward shaping for multi-agent systems. Tumar K, Yolum P, Sonnenberg L, Stone P, eds. 10th Internat. Conf. Autonomous Agents Multiagent Systems, vol. 1 (International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC), 225–232.Google Scholar
• Dong Y, Koster RD, Roy D, Yu Y (2022) Dynamic vehicle allocation policies for shared autonomous electric fleets. Transportation Sci. 56:1238–1258.Link, Google Scholar
• Fagnant DJ, Kockelman KM (2014) The travel and environmental implications of shared autonomous vehicles, using agent-based model scenarios. Transportation Res. Part C: Emerging Tech. 40:1–13.Crossref, Google Scholar
• Florio AM, Absi N, Feillet D (2021) Routing electric vehicles on congested street networks. Transportation Sci. 55(1):238–256.Link, Google Scholar
• Froger A, Jabali O, Mendoza JE, Laporte G (2022) The electric vehicle routing problem with capacitated charging stations. Transportation Sci. 56(2):460–482.Link, Google Scholar
• Fuel Economy Guide (2020) Accessed December 1, 2021, https://www.fueleconomy.gov/feg/pdfs/guides/FEG2020.pdf.Google Scholar
• Guillet M, Hiermann G, Kröller A, Schiffer M (2022) Electric vehicle charging station search in stochastic environments. Transportation Sci. 56(2):483–500.Link, Google Scholar
• He L, Ma G, Qi W, Wang X (2021) Charging an electric vehicle-sharing fleet. Manufacturing Service Oper. Management 23(2):471–487.Abstract, Google Scholar
• Ho SC, Szeto WY, Kuo YH, Leung JM, Petering M, Tou TW (2018) A survey of dial-a-ride problems: Literature review and recent developments. Transportation Res. Part B: Methodological 111:395–421.Crossref, Google Scholar
• Hyland M, Mahmassani HS (2018) Dynamic autonomous vehicle fleet operations: Optimization-based strategies to assign AVs to immediate traveler demand requests. Transportation Res. Part C: Emerging Tech. 92:278–297.Crossref, Google Scholar
• Iacobucci R, McLellan B, Tezuka T (2019) Optimization of shared autonomous electric vehicles operations with charge scheduling and vehicle-to-grid. Transportation Res. Part C: Emerging Tech. 100:34–52.Crossref, Google Scholar
• Jian L, Zheng Y, Shao Z (2017) High efficient valley-filling strategy for centralized coordinated charging of large-scale electric vehicles. Appl. Energy 186:46–55.Crossref, Google Scholar
• Kahlen MT, Ketter W, van Dalen J (2018) Electric vehicle virtual power plant dilemma: Grid balancing vs. customer mobility. Production Oper. Management 27(11):2054–2070.Google Scholar
• Kulkarni TD, Narasimhan K, Saeedi A, Tenenbaum J (2016) Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation. Garnett R, Sugiyama M, Guyon I, Lee DD, von Luxburg U, eds. Proc. 30th Internat. Conf. Neural Inform. Processing Systems (Curran Associates Inc., Red Hook, NY), 3682–3690.Google Scholar
• Kullman ND, Goodson JC, Mendoza JE (2021b) Electric vehicle routing with public charging stations. Transportation Sci. 55(3):637–659.Link, Google Scholar
• Kullman ND, Cousineau M, Goodson JC, Mendoza JE (2021a) Dynamic ride-hailing with electric vehicles. Transportation Sci. 56(3):775–794.Link, Google Scholar
• Levin MW, Odell M, Samarasena S, Schwartz A (2019) A linear program for optimal integration of shared autonomous vehicles with public transit. Transportation Res. Part C: Emerging Tech. 109:267–288.Crossref, Google Scholar
• Li M, Qin Z, Jiao Y, Yang Y, Wang J, Wang C, Wu G, et al. (2019) Efficient ridesharing order dispatching with mean field multi-agent reinforcement learning. The World Wide Web Conf., 983–994.Google Scholar
• Liang Y, Ding Z, Ding T, Lee WJ (2020) Mobility-aware charging scheduling for shared on-demand electric vehicle fleet using deep reinforcement learning. IEEE Trans. Smart Grid 12(2):1380–1393.Crossref, Google Scholar
• Littman ML (1994) Markov games as a framework for multi-agent reinforcement learning. Proc. 11th Internat. Conf. Machine Learn. (Morgan Kaufmann Publishers Inc., San Francisco), 157–163.Crossref, Google Scholar
• Loeb B, Kockelman KM, Liu J (2018) Shared autonomous electric vehicle (SAEV) operations across the Austin, Texas network with charging infrastructure decisions. Transportation Res. Part C: Emerging Tech. 89:222–233.Crossref, Google Scholar
• Lokhandwala M, Cai H (2018) Dynamic ride sharing using traditional taxis and shared autonomous taxis: A case study of NYC. Transportation Res. Part C: Emerging Tech. 97:45–60.Crossref, Google Scholar
• Lokhandwala M, Cai H (2020) Siting charging stations for electric vehicle adoption in shared autonomous fleets. Transportation Res. Part D: Transport Environ. 80:102231.Crossref, Google Scholar
• Ma TY, Xie S (2021) Optimal fast charging station locations for electric ridesharing with vehicle-charging station assignment. Transportation Res. Part D Transportation Environ. 90:102682.Crossref, Google Scholar
• Mahmassani HS (2016) 50th anniversary invited article—Autonomous vehicles and connected vehicle systems: Flow and operations considerations. Transportation Sci. 50(4):1140–1162.Link, Google Scholar
• Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, et al. (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533.Crossref, Google Scholar
• Open Charge Map (2020) https://www.openchargemap.org/.Google Scholar
• Oroojlooyjadid A, Nazari M, Snyder LV, Takáč M (2022) A deep Q-network for the beer game: Deep reinforcement learning for inventory optimization. Manufacturing Service Oper. Management 24(1):285–304.Link, Google Scholar
• Pantelidis TP, Li L, Ma TY, Chow JY, Jabari SEG (2022) A node-charge graph-based online carshare rebalancing policy with capacitated electric charging. Transportation Sci. 56(3):654–676.Link, Google Scholar
• Qi W, Sha M, Li S (2022) When shared autonomous electric vehicles meet microgrids: Citywide energy-mobility orchestration. Manufacturing Service Oper. Management 24:2389–2406.Link, Google Scholar
• Qin Z, Tang X, Jiao Y, Zhang F, Xu Z, Zhu H, Ye J (2020) Ride-hailing order dispatching at DiDi via reinforcement learning. INFORMS J. Appl. Analytics 50(5):272–286.Link, Google Scholar
• Roni MS, Yi Z, Smart JG (2019) Optimal charging management and infrastructure planning for free-floating shared electric vehicles. Transportation Res. Part D: Transport Environ. 76:155–175.Crossref, Google Scholar
• Rossi F, Zhang R, Hindy Y, Pavone M (2018) Routing autonomous vehicles in congested transportation networks: Structural properties and coordination algorithms. Autonomous Robots 42(7):1427–1442.Crossref, Google Scholar
• Schroer K, Ketter W, Lee TY, Gupta A, Kahlen M (2022) Data-driven competitor-aware positioning in on-demand vehicle rental networks. Transportation Sci. 56(1):182–200.Link, Google Scholar
• Shi J, Gao Y, Wang W, Yu N, Ioannou PA (2020) Operating electric vehicle fleet for ride-hailing services with reinforcement learning. IEEE Trans. Intelligent Transportation Systems 21(11):4822–4834.Crossref, Google Scholar
• Shou Z, Di X (2020) Reward design for driver repositioning using multi-agent reinforcement learning. Transportation Res., Part C Emerging Tech. 119:102738.Crossref, Google Scholar
• Siddiq A, Taylor TA (2022) Ride-hailing platforms: Competition and autonomous vehicles. Manufacturing Service Oper. Management 24:1511–1528.Link, Google Scholar
• Sperling D (2018) Three Revolutions: Steering Automated, Shared, and Electric Vehicles to a Better Future (Island Press).Crossref, Google Scholar
• Sutton RS, Barto AG (2018) Reinforcement Learning: An Introduction (MIT Press, Cambridge, MA).Google Scholar
• Sutton RS, Precup R, Singh S (1999) Between MDPS and semi-MDPS: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112(1–2):181–211.Crossref, Google Scholar
• Sweda TM, Dolinskaya IS, Klabjan D (2017) Optimal recharging policies for electric vehicles. Transportation Sci. 51(2):457–459.Link, Google Scholar
• Valogianni K, Ketter W, Collins J, Zhdanov D (2020) Sustainable electric vehicle charging using adaptive pricing. Production Oper. Management 29(6):1550–1572.Google Scholar
• Wang E, Ding R, Yang Z, Jin H, Miao C, Su L, Zhang F, Qiao C, Wang X (2022) Joint charging and relocation recommendation for e-taxi drivers via multi-agent mean field hierarchical reinforcement learning. IEEE Trans. Mobile Comput. 21(4):1274–1290.Google Scholar
• Wu OQ, Yücel S, Zhou YH (2022) Smart charging of electric vehicles: An innovative business model for utility firms. Manufacturing Service Oper. Management 24:2481–2499.Link, Google Scholar
• Yang Y, Luo R, Li M, Zhou M, Zhang W, Wang J (2018) Mean field multi-agent reinforcement learning. Dy J, Krause A, eds. Proc. 35th Internat. Conf. Machine Learn., vol. 80, Proceedings of Machine Learning Research Series (PMLR), 5571–5580.Google Scholar
• Zhang TZ, Chen TD (2020) Smart charging management for shared autonomous electric vehicle fleets: A Puget Sound case study. Transportation Res. Part D: Transport Environ. 78:102184.Crossref, Google Scholar
• Zhu Z, Lambotharan S, Chin WH, Fan Z (2016) A mean field game theoretic approach to electric vehicles charging. IEEE Access 4:3501–3510.Crossref, Google Scholar