Convergence and Inference of Stream Stochastic Gradient Descent, with Applications to Queueing Systems and Inventory Control

Stream stochastic gradient descent (SGD) is a simple and efficient method for solving online optimization problems in operations research (OR), where data are generated by parameter-dependent Markov chains. Unlike traditional approaches which require increasing batch sizes during iterations, stream SGD uses a single sample per iteration, significantly improving sample efficiency. This paper establishes a systematic framework for analyzing stream SGD, leveraging the Poisson equation solution to address gradient bias and statistical dependence. We prove optimal $\mathcal{O}(T^{-1})$ convergence rates and the state-of-the-art $\mathcal{O}(\log T)$ regret, while also introducing an online inference method for uncertainty quantification and supporting it by a novel functional central limit theorem. We propose a novel Wasserstein-type divergence to describe the framework’s conditions, which makes the assumptions in question directly verified via coupling techniques tailored to underlying OR models. We consider applications in queueing systems and inventory management, demonstrating the practicality and broad relevance, as well as providing new insights into the effectiveness of stream SGD in OR fields.

Funding: Financial support from the National Natural Science Foundation of China [Grants 12271011, 12350001, and 72171205] and the National Key Research and Development Program of China [Grants 2022YFA1004002 and 2024YFA1014203] is gratefully acknowledged.

Supplemental Material: All supplemental materials, including the code, data, and files required to reproduce the results, are available at https://doi.org/10.1287/opre.2025.1662.