Stochastic Approximation Proximal Method of Multipliers for Convex Stochastic Programming

References

• [1] Asi H, Duchi JC (2019a) The importance of better models in stochastic optimization. Proc. Natl. Acad. Sci. USA 116(46):22924–22930.Crossref, Google Scholar
• [2] Asi H, Duchi JC (2019b) Stochastic (approximate) proximal point methods: Convergence, optimality, and adaptivity. SIAM J. Optim. 29(3):2257–2290.Crossref, Google Scholar
• [3] Boob D, Deng Q, Lan G (2019) Stochastic first-order methods for convex and nonconvex functional constrained optimization. Preprint, submitted August 7, https://arxiv.org/abs/1908.02734.Google Scholar
• [4] Chung KL (1954) On a stochastic approximation method. Ann. Math. Statist. 25(3):463–483.Crossref, Google Scholar
• [5] Ghadimi S, Lan G (2012) Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization I: A generic algorithmic framework. SIAM J. Optim. 22(4):1469–1492.Crossref, Google Scholar
• [6] Ghadimi S, Lan G (2013a) Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization, II: Shrinking procedures and optimal algorithms. SIAM J. Optim. 23(4):2061–2089.Crossref, Google Scholar
• [7] Ghadimi S, Lan G (2013b) Stochastic first- and zeroth-order methods for nonconvex stochastic programming. SIAM J. Optim. 23(4):2341–2368.Crossref, Google Scholar
• [8] Ghadimi S, Lan G, Zhang H (2016) Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization. Math. Programming 155(1-2):267–305.Crossref, Google Scholar
• [9] Lan G (2012) An optimal method for stochastic composite optimization. Math. Programming 133(1-2):365–397.Crossref, Google Scholar
• [10] Lan G (2016) Gradient sliding for composite optimization. Math. Programming 159(1-2):201–235.Crossref, Google Scholar
• [11] Lan G (2020) First-Order and Stochastic Optimization Methods for Machine Learning, Springer Series in the Data Sciences (Springer, Cham, Switzerland).Crossref, Google Scholar
• [12] Lan G, Zhou Z (2020) Algorithms for stochastic optimization with function or expectation constraints. Comput. Optim. Appl. 76(2):461–498.Crossref, Google Scholar
• [13] Lan G, Nemirovski A, Shapiro A (2012) Validation analysis of mirror descent stochastic approximation method. Math. Programming 134(2):425–458.Crossref, Google Scholar
• [14] Lin Q, Nadarajah S, Soheili N, Yang T (2020) A data efficient and feasible level set method for stochastic convex optimization with expectation constraints. J. Machine Learn. Res. 21(143):1–45.Google Scholar
• [15] Nemirovski A, Juditsky A, Lan G, Shapiro A (2009) Robust stochastic approximation approach to stochastic programming. SIAM J. Optim. 19(4):1574–1609.Crossref, Google Scholar
• [16] Nesterov Y (2018) Lectures on Convex Optimization, 2nd ed., Springer Optimization and Its Applications, vol. 137 (Springer, Cham, Switzerland).Crossref, Google Scholar
• [17] Pflug GC (1996) Optimization of Stochastic Models: The Interface Between Simulation and Optimization, The Kluwer International Series in Engineering and Computer Science, vol. 373 (Kluwer Academic Publishers, Boston).Crossref, Google Scholar
• [18] Polyak BT (1990) New stochastic approximation type procedures. Avtomat. i Telemekh. 7:98–107.Google Scholar
• [19] Polyak BT, Juditsky AB (1992) Acceleration of stochastic approximation by averaging. SIAM J. Control Optim. 30(4):838–855.Crossref, Google Scholar
• [20] Robbins H, Monro S (1951) A stochastic approximation method. Ann. Math. Statist. 22:400–407.Crossref, Google Scholar
• [21] Rockafellar RT (1976) Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1(2):97–116.Link, Google Scholar
• [22] Rockafellar RT (1976) Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5):877–898.Crossref, Google Scholar
• [23] Sacks J (1958) Asymptotic distribution of stochastic approximation procedures. Ann. Math. Statist. 29:373–405.Crossref, Google Scholar
• [24] Shalev-Shwartz S (2011) Online learning and online convex optimization. Foundations Trends® Machine Learn. 4(2):107–194.Crossref, Google Scholar
• [25] Xiao L (2010) Dual averaging methods for regularized stochastic learning and online optimization. J. Machine Learn. Res. 11:2543–2596.Google Scholar
• [26] Yu H, Neely MJ, Wei X (2017) Online convex optimization with stochastic constraints. von Luxburg U, Guyon I, Bengio S, Wallach H, Fergus R, eds. Proc. 31st Internat. Conf. Neural Inform. Processing Systems (Curran Associates, Red Hook, NY), 1428–1438.Google Scholar