Multiagent Environments for Vehicle Routing Problems

Research on reinforcement learning (RL) approaches for discrete optimization problems has increased considerably, extending RL to areas classically dominated by operations research (OR). Vehicle routing problems are a good example of discrete optimization problems with high practical relevance for which RL techniques have achieved notable success. Despite these advances, open-source development frameworks remain scarce, hindering both algorithm testing and objective comparison of results. This situation ultimately slows down progress in the field and limits the exchange of ideas between the RL and OR communities. Here, we propose MAEnvs4VRP library, a unified framework for multiagent vehicle routing environments that supports classical, dynamic, stochastic, and multitask problem variants within a single modular design. The library, built on PyTorch, provides a flexible and modular architecture design that facilitates customization and the incorporation of new routing problems. It follows the agent environment cycle (“AEC”) games model and features an intuitive API, enabling rapid adoption and seamless integration into existing reinforcement learning frameworks.

History: Accepted by Ted Ralphs, Area Editor for Software Tools.

Funding: R. Gama gratefully acknowledges the Research Centre in Digital Services (CISeD), the Instituto Politécnico de Viseu, and the Foundation for Science and Technology, I.P. (FCT) for their support during the work [Grants UIDB/05583/2020 and 2023.13303.CPCA.A0].

Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information (https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2025.1211) as well as from the IJOC GitHub software repository (https://github.com/INFORMSJoC/2025.1211). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/.