Randomized Subspace Gradient Method for Constrained Optimization

References

• [1] Achiam J, Held D, Tamar A, Abbeel P (2017) Constrained policy optimization. Precup D, Teh YW, eds. Proc. 34th Internat. Conf. Machine Learn., Proceedings of Machine Learning Research, vol. 70 (PMLR, New York), 22–31.Google Scholar
• [2] Berglund E, Khirirat S, Wang X (2022) Zeroth-order randomized subspace newton methods. Proc. 2022 IEEE Internat. Conf. Acoustics, Speech, and Signal Processing (ICASSP) (IEEE, Piscataway, NJ), 6002–6006.Google Scholar
• [3] Cartis C, Fowkes J, Shao Z (2022) Randomised subspace methods for non-convex optimization, with applications to nonlinear least-squares. Preprint, submitted November 17, https://arxiv.org/abs/2211.09873.Google Scholar
• [4] Cartis C, Massart E, Otemissov A (2023) Bound-constrained global optimization of functions with low effective dimensionality using multiple random embeddings. Math. Programming 198(1):997–1058.Crossref, Google Scholar
• [5] Cartis C, Massart E, Otemissov A (2023) Global optimization using random embeddings. Math. Programming 200(2):781–829.Crossref, Google Scholar
• [6] Chen L, Hu X, Wu H (2020) Randomized fast subspace descent methods. Preprint, submitted June 11, https://arxiv.org/abs/2006.06589.Google Scholar
• [7] Du DZ, Wu F, Zhang XS (1990) On Rosen’s gradient projection methods. Ann. Oper. Res. 24(1):9–28.Crossref, Google Scholar
• [8] Fuji T, Poirion P, Takeda A (2025) Randomized subspace regularized Newton method for unconstrained non-convex optimization. Open J. Math. Optim. 6(8):1–35.Google Scholar
• [9] Gao Z, Lai Y, Hu Z (1996) A generalized gradient projection method for optimization problems with equality and inequality constraints about arbitrary initial point. Acta Mathematicae Applicatae Sinica 12:40–49.Crossref, Google Scholar
• [10] Gong C, Liu X, Liu Q (2021) Automatic and harmless regularization with constrained and lexicographic optimization: A dynamic barrier approach. Ranzato M, Beygelzimer A, Dauphin Y, Liang PS, Wortman Vaughan J, eds. Adv. Neural Inform. Processing Systems, vol. 34 (Curran Associates, Inc., Red Hook, NY), 29630–29642.Google Scholar
• [11] Goulart PJ, Chen Y (2024) Clarabel: An interior-point solver for conic programs with quadratic objectives. Preprint, submitted May 21, https://arxiv.org/abs/2405.12762.Google Scholar
• [12] Gower R, Kovalev D, Lieder F, Richtárik P (2019) RSN: Randomized subspace Newton. Wallach H, Larochelle H, Beygelzimer A, d’Alché-Buc F, Fox E, Garnett R, eds. Adv. Neural Inform. Processing Systems, vol. 32 (Curran Associates, Inc., Red Hook, NY).Google Scholar
• [13] Griewank A, Walther A (2008) Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, 2nd ed. (SIAM, Philadelphia).Google Scholar
• [14] Hanzely F, Doikov N, Nesterov Y, Richtarik P (2020) Stochastic subspace cubic Newton method. Daumé H III, Singh A, eds. Proc. 37th Internat. Conf. Machine Learn., Proceedings of Machine Learning Research, vol. 119 (PMLR, New York), 4027–4038.Google Scholar
• [15] Johnson W, Lindenstrauss J (1984) Extensions of Lipschitz mappings into a Hilbert space. Hedlund G, ed. Conf. Modern Anal. Probab., Contemporary Mathematics, vol. 26 (American Mathematical Society, Providence, RI), 189–206.Google Scholar
• [16] Komiyama J, Takeda A, Honda J, Shimao H (2018) Nonconvex optimization for regression with fairness constraints. Dy J, Krause A, eds. Proc. 35th Internat. Conf. Machine Learn., Proceedings of Machine Learning Research, vol. 80 (PMLR, New York), 2737–2746.Google Scholar
• [17] Kozak D, Becker S, Doostan A, Tenorio L (2021) A stochastic subspace approach to gradient-free optimization in high dimensions. Comput. Optim. Appl. 79(2):339–368.Crossref, Google Scholar
• [18] Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791.Crossref, Google Scholar
• [19] Margossian CC (2019) A review of automatic differentiation and its efficient implementation. Wiley Interdisciplinary Rev. Data Mining Knowledge Discovery 9(4):e1305.Crossref, Google Scholar
• [20] Moldovan TM, Abbeel P (2012) Safe exploration in Markov decision processes. Langford J, Pineau J, eds. Proc. 29th Internat. Conf. Machine Learn. (Omnipress, Edinburgh, UK), 1451–1458.Google Scholar
• [21] Rosen JB (1960) The gradient projection method for nonlinear programming. Part I. Linear constraints. J. Soc. Indust. Appl. Math. 8(1):181–217.Crossref, Google Scholar
• [22] Rosen JB (1961) The gradient projection method for nonlinear programming. Part II. Nonlinear constraints. J. Soc. Indust. Appl. Math. 9(4):514–532.Crossref, Google Scholar
• [23] Tibshirani R, Saunders M, Rosset S, Zhu J, Knight K (2005) Sparsity and smoothness via the fused lasso. J. Roy. Statist. Soc. Ser. B Statist. Methodology 67(1):91–108.Crossref, Google Scholar
• [24] Vershynin R (2018) High-Dimensional Probability: An Introduction with Applications in Data Science, vol. 47 (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• [25] Wang W, Hua S, Tang J (2013) A generalized gradient projection filter algorithm for inequality constrained optimization. J. Appl. Math. 2013(1):1–6.Google Scholar
• [26] Zafar MB, Valera I, Rodriguez MG, Gummadi KP (2017) Fairness constraints: Mechanisms for fair classification. Singh A, Zhu J, eds. Proc. 20th Internat. Conf. Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, vol. 54 (PMLR, New York), 962–970.Google Scholar