Semibandit Learning for Monotone Stochastic Optimization

Stochastic optimization is a widely used approach for optimization under uncertainty, when uncertain input parameters are modeled by random variables. Exact or approximation algorithms have been obtained for several fundamental problems in this area. However, a significant limitation of this approach is that it requires full knowledge of the underlying probability distributions. Can we still get good (approximation) algorithms if these distributions are unknown and the algorithm needs to learn them through repeated interactions? In this paper, we resolve this question for a large class of monotone stochastic problems by providing a generic online learning algorithm with $\sqrt{T\text{log\hspace{0.17em}}T}$ regret relative to the best approximation algorithm (under known distributions). Importantly, our online algorithm works in a semibandit setting, in which, in each period, the algorithm only observes samples from the random variables that were actually probed. Moreover, our result extends to settings with censored and binary feedback, in which the policy only observes truncated or thresholded versions of the probed variables. Our framework applies to several fundamental problems, such as prophet inequality, Pandora’s box, stochastic knapsack, single-resource revenue management, and sequential posted pricing.

Funding: This work was supported by the National Science Foundation Division of Computing and Communication Foundations [Grant 2418495].

Supplemental Material: The online appendix is available at https://doi.org/10.1287/opre.2025.2250.