Solving Stochastic Optimization with Expectation Constraints Efficiently by a Stochastic Augmented Lagrangian-Type Algorithm

References

• Akhtar Z, Bedi AS, Rajawat K (2021) Conservative stochastic optimization with expectation constraints. IEEE Trans. Signal Process. 69:3190–3205.Crossref, Google Scholar
• Bai X, Sun J, Zheng X (2021) An augmented Lagrangian decomposition method for chance-constrained optimization problems. INFORMS J. Comput. 33(3):1056–1069.Link, Google Scholar
• Beck A (2017) First-Order Methods in Optimization, MOS-SIAM Series on Optimization, vol. 25 (Society for Industrial and Applied Mathematics, Philadelphia).Crossref, Google Scholar
• Bottou L, Curtis FE, Nocedal J (2018) Optimization methods for large-scale machine learning. SIAM Rev. 60(2):223–311.Crossref, Google Scholar
• ChaLearn (2008) CINA is an econometrics dataset. Accessed April 3, 2008, http://www.causality.inf.ethz.ch/data/CINA.html.Google Scholar
• Cao X, Zhang J, Poor HV (2021) Constrained online convex optimization with feedback delays. IEEE Trans. Automatic Control 66(11):5049–5064.Crossref, Google Scholar
• Dentcheva D, Ruszczyński A (2003) Optimization with stochastic dominance constraints. SIAM J. Optim. 14(2):548–566.Crossref, Google Scholar
• Dentcheva D, Martinez G, Wolfhagen E (2016) Augmented Lagrangian methods for solving optimization problems with stochastic-order constraints. Oper. Res. 64(6):1451–1465.Link, Google Scholar
• Guyon I, Gunn S, Ben-Hur A, Dror G (2004) Result analysis of the NIPS 2003 Feature Selection Challenge. Saul L, Weiss Y, Bottou L, eds. Advances in Neural Information Processing Systems, vol. 17 (MIT Press, Cambridge, MA), 545–552.Google Scholar
• Kallio M, Dehghan Hardoroudi N (2018) Second-order stochastic dominance constrained portfolio optimization: Theory and computational tests. Eur. J. Oper. Res. 264(2):675–685.Crossref, Google Scholar
• Keçeci NF, Kuzmenko V, Uryasev S (2016) Portfolios dominating indices: Optimization with second-order stochastic dominance constraints vs. minimum and mean variance portfolios. J. Risk Financial Management 9(4):1–14.Google Scholar
• Lan G (2016) Gradient sliding for composite optimization. Math. Programming 159(1–2):201–235.Google Scholar
• Lan G (2020) First-Order and Stochastic Optimization Methods for Machine Learning, Springer Series in the Data Sciences (Springer, Cham, Switzerland).Crossref, Google Scholar
• Lan G, Zhou Z (2020) Algorithms for stochastic optimization with function or expectation constraints. Comput. Optim. Appl. 76(2):461–498.Crossref, Google Scholar
• LeCun Y, Cortes C, Burges CJC (1998) The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/.Google Scholar
• Lin Q, Nadarajah S, Soheili N, Yang T (2020) A data efficient and feasible level set method for stochastic convex optimization with expectation constraints. J. Machine Learn. Res. 21(143):1–45.Google Scholar
• Liu A, Lau VKN, Kananian B (2019) Stochastic successive convex approximation for non-convex constrained stochastic optimization. IEEE Trans. Signal Process. 67(16):4189–4203.Crossref, Google Scholar
• Liu H, Xiao X, Zhang L (2022) Augmented Lagrangian methods for time-varying constrained online convex optimization. Preprint, submitted May 19, https://arxiv.org/abs/2205.09571.Google Scholar
• Nemirovski A, Juditsky A, Lan G, Shapiro A (2009) Robust stochastic approximation approach to stochastic programming. SIAM J. Optim. 19(4):1574–1609.Crossref, Google Scholar
• Noyan N (2018) Risk-averse stochastic modeling and optimization. Gel E, Ntaimo L, eds. Recent Advances in Optimization and Modeling of Contemporary Problems, INFORMS TutORials in Operations Research (INFORMS, Catonsville, MD), 221–254.Link, Google Scholar
• Parpas P, Rustem B (2007) Computational assessment of nested Benders and augmented Lagrangian decomposition for mean-variance multistage stochastic problems. INFORMS J. Comput. 19(2):239–247.Link, Google Scholar
• Pflug GC (1996) Optimization of Stochastic Models: The Interface Between Simulation and Optimization, Kluwer International Series in Engineering and Computer Science, vol. 373 (Kluwer Academic Publishers, Boston).Crossref, Google Scholar
• Polyak BT (1967) A general method of solving extremum problems. Dokl. Akad. Nauk SSSR 174(1):593–597.Google Scholar
• Polyak BT (1990) New stochastic approximation type procedures. [In Russian.] Avtomatika Telemekhanika 51(7, Part 2):98–107.Google Scholar
• Polyak BT, Juditsky AB (1992) Acceleration of stochastic approximation by averaging. SIAM J. Control Optim. 30(4):838–855.Crossref, Google Scholar
• Robbins H, Monro S (1951) A stochastic approximation method. Ann. Math. Statist. 22(3):400–407.Crossref, Google Scholar
• Rockafellar RT (1976) Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1(2):97–116.Link, Google Scholar
• Rockafellar RT, Uryasev S (2000) Optimization of conditional value-at-risk. J. Risk 2(3):21–41.Crossref, Google Scholar
• Römisch W (2003) Stability of stochastic programming problems. Ruszczyński A, Shapiro A, eds. Stochastic Programming, Handbooks in Operations Research and Management Science, vol. 10(Elsevier, Amsterdam), 483–554.Crossref, Google Scholar
• Ruszczyński A, Shapiro A (2003) Stochastic programming models. Ruszczyński A, Shapiro A, eds. Stochastic Programming, Handbooks in Operations Research and Management Science, vol. 10 (Elsevier, Amsterdam), 1–64.Crossref, Google Scholar
• Scott C, Nowak R (2005) A Neyman-Pearson approach to statistical learning. IEEE Trans. Inform. Theory 51(11):3806–3819.Crossref, Google Scholar
• Tong X, Feng Y, Zhao A (2016) A survey on Neyman-Pearson classification and suggestions for future research. Wiley Interdisciplinary Rev. Comput. Statist. 8(2):64–81.Crossref, Google Scholar
• Wang W, Lu C (2015) Projection onto the capped simplex. Preprint, submitted March 3, https://arxiv.org/abs/1503.01002.Google Scholar
• Xiao X (2019) Penalized stochastic gradient methods for stochastic convex optimization with expectation constraints. Preprint, submitted September 12, http://www.optimization-online.org/DB_FILE/2019/09/7364.pdf.Google Scholar
• Yan Y, Xu Y (2022) Adaptive primal-dual stochastic gradient method for expectation-constrained convex stochastic programs. Math. Programming Comput. 14(2):319–363.Crossref, Google Scholar
• Yu H, Neely MJ, Wei X (2017) Online convex optimization with stochastic constraints. Guyon I, Von Luxburg U, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R, eds. Advances in Neural Information Processing Systems, vol. 30 (Curran Associates, Red Hook, NY), 1428–1438.Google Scholar
• Zhang L, Zhang Y, Wu J (2020) Stochastic approximation proximal method of multipliers for convex stochastic programming. Preprint, submitted September 13, https://arxiv.org/abs/1907.12226.Google Scholar
• Zhao XY, Sun D, Toh KC (2010) A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim. 20(4):1737–1765.Crossref, Google Scholar