Inpatient Overflow Management with Proximal Policy Optimization

Problem definition: Managing inpatient flow in large hospital systems is challenging due to the complexity of assigning randomly arriving patients: either waiting for primary units or being overflowed to alternative units. Current practices rely on ad hoc rules, whereas prior analytical approaches struggle with the intractably large state and action spaces inherent in patient-unit matching. Scalable decision support is needed to optimize overflow management while accounting for time-periodic fluctuations in patient flow. Methodology/results: We develop a scalable decision-making framework using proximal policy optimization (PPO) to optimize overflow decisions in a time-periodic, long-run average cost setting. To address the combinatorial complexity, we introduce atomic actions, which decompose multipatient routing into sequential assignments. We further enhance computational efficiency through a partially shared policy network designed to balance parameter sharing with time-specific policy adaptations and a queueing-informed value function approximation to improve policy evaluation. Our method significantly reduces the need for extensive simulation data, a common limitation in reinforcement learning applications. Case studies on hospital systems with up to 20 patient classes and 20 wards demonstrate that our approach matches or outperforms existing benchmarks, including approximate dynamic programming, which is computationally infeasible beyond five wards. Managerial implications: Our framework offers a scalable, efficient, and explainable solution for managing patient flow in complex hospital systems. More broadly, our results highlight that domain-aware adaptation is more critical to improving algorithm performance than fine-tuning neural network parameters when applying general-purpose algorithms to specific applications.

Supplemental Material: The online appendices and code files are available at https://doi.org/10.1287/msom.2025.0215 and https://github.com/OverflowPPO/InpatientOverflow.