Nonconvex Lagrangian-Based Optimization: Monitoring Schemes and Global Convergence

References

• Attouch H, Bolte J (2009) On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math. Programming 116(1):5–16.Crossref, Google Scholar
• Attouch H, Bolte J, Svaiter BF (2013) Convergence of descent methods for semi-algebraic and tame problems: Proximal algorithms, forward backward splitting, and regularized Gauss-Seidel methods. Math. Programming 137(1–2):91–129.Crossref, Google Scholar
• Auslender A (2015) An exact penalty method for nonconvex problems covering, in particular, nonlinear programming, semidefinite programming, and second-order cone programming. SIAM J. Optim. 25(3):1732–1759.Crossref, Google Scholar
• Beck A, Eldar YC (2013) Sparsity constrained nonlinear optimization: Optimality conditions and algorithms. SIAM J. Optim. 23(3):1480–1509.Crossref, Google Scholar
• Beck A, Hallak N (2016) On the minimization over sparse symmetric sets: Projections, optimality conditions, and algorithms. Math. Oper. Res. 41(1):196–223.Link, Google Scholar
• Becker S, Bobin J, Candès E (2011) NESTA: A fast and accurate first-order method for sparse recovery. SIAM J. Imaging Sci. 4(1):1–39.Crossref, Google Scholar
• Bertsekas DP (1996) Constrained Optimization and Lagrange Multiplier Methods (Athena Scientific, Belmont MA).Google Scholar
• Bertsekas DP, Tsitsiklis JN (1989) Parallel and Distributed Computation: Numerical Methods (Prentice-Hall, Enlewood Cliffs, NJ).Google Scholar
• Björck A (1996) Numerical Methods for Least Squares Problems (SIAM, Philadelphia).Crossref, Google Scholar
• Bolte J, Pauwels E (2016) Majorization-minimization procedures and convergence of SQP methods for semi-algebraic and tame programs. Math. Oper. Res. 41(2):442–465.Link, Google Scholar
• Bolte J, Daniilidis A, Lewis AS (2007) The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM J. Optim. 17(4):1205–1223.Crossref, Google Scholar
• Bolte J, Sabach S, Teboulle M (2014) Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Programming 146(1–2):459–494.Crossref, Google Scholar
• Chavent G (2009) Nonlinear Least Squares for Inverse Problems (Springer, Dordrecht, Netherlands).Google Scholar
• Edelman A, Arias TA, Steven ST (1998) The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2):303–353.Crossref, Google Scholar
• Fortin M, Glowinski R (1983) Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Valued Problems (Elsevier, Amsterdam).Google Scholar
• Glowinski R, Le Tallec P (1989) Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics (SIAM, Philadelphia).Crossref, Google Scholar
• Hesse R, Luke DR (2013) Nonconvex notions of regularity and convergence of fundamental algorithms for feasibility problems. SIAM J. Optim. 23(4):2397–2419.Crossref, Google Scholar
• Haarhoff PC, Buys JD (1970) A new method for the optimization of a nonlinear function subject to nonlinear constraints. Computer J. 13(2):178–184.Crossref, Google Scholar
• Hestenes M (1969) Multiplier and gradient methods. J. Optim. Theory Appl. 4(5):303–320.Crossref, Google Scholar
• Kim S, Kojima M (2009) Solving polynomial least squares problems via semidefinite programming relaxations. J. Global Optim. 46(1):1–23.Crossref, Google Scholar
• Kurdyka K (1998) On gradients of functions definable in o-minimal structures. Ann. Inst. Fourier 48(3):769–783.Crossref, Google Scholar
• Lewis AS, Luke DR, Malick J (2009) Local linear convergence for alternating and averaged nonconvex projections. Foundations of Comput. Math. 9(4):485–513.Crossref, Google Scholar
• Li G, Pong TK (2015) Global convergence of splitting methods for nonconvex composite optimization. SIAM J. Optim. 25(4):2434–2460.Crossref, Google Scholar
• Łojasiewicz S (1963) Une propriété topologique des sous-ensembles analytiques réels. Les Équations aux Derivées Partielles (Éditions du Centre National de la Recherche Scientifique, Paris), 87–89.Google Scholar
• Luss R, Teboulle M (2013) Conditional gradient algorithms for rank-one matrix approximations with a sparsity constraint. SIAM Rev. 55(1):65–98.Crossref, Google Scholar
• Milnor J (1997) Topology from the Differentiable Viewpoint (Princeton University Press, Princeton, NJ).Google Scholar
• Ortega JM, Rheinboldt WC (1970) Iterative Solution of Nonlinear Equations in Several Variables (Academic Press, New York).Google Scholar
• Powell MJD (1969) A method for nonlinear constraints in minimization problems. Fletcher R, ed. Optimization (Academic Press, New York), 283–298.Google Scholar
• Rockafellar RT (1976) Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1(2):97–116.Link, Google Scholar
• Rockafellar RT, Wets RJ-B (1998) Variational Analysis. Grundlehren der Mathematischen Wissenschaften, Vol. 317 (Springer, Berlin).Crossref, Google Scholar
• Shefi R, Teboulle M (2014) Rate of convergence analysis of decomposition methods based on the proximal method of multipliers for convex minimization. SIAM J. Optim. 24(1):269–297.Crossref, Google Scholar
• Sra S, Nowozin S, Wright SJ (2011) Optimization for Machine Learning (The MIT Press, Cambridge, MA).Crossref, Google Scholar