Open Problem—Adaptive Constant-Step Stochastic Approximation

References

• Bottou L, Curtis F, Nocedal J (2018) Optimization methods for large-scale machine learning. SIAM Rev. 60(2):223–311.Google Scholar
• Bubeck S (2015) Convex Optimization: Algorithms and Complexity, Foundations and Trends in Machine Learning, vol. 8 (Now Publishers Inc., Boston).Google Scholar
• Dieuleveut A, Durmus A, Bach F (2017) Bridging the gap between constant step size stochastic gradient descent and Markov chains. Preprint, submitted July 20, arXiv:1707.06386.Google Scholar
• Fabian V (1968) On asymptotic normality in stochastic approximation. Ann. Math. Statist. 39(4):1327–1332.Google Scholar
• Kushner HJ, Yin GG (2003) Stochastic Approximation and Recursive Algorithms and Applications (Springer-Verlag, New York).Google Scholar
• Moritz P, Nishihara R, Jordan MI (2015) A linearly-convergent stochastic L-BFGS algorithm. Preprint, submitted August 9, arXiv:1508.02087.Google Scholar
• Nocedal J, Wright SJ (2006) Numerical Optimization (Springer-Verlag, Berlin).Google Scholar
• Pilanci M, Wainwright MJ (2015) Newton sketch: A linear-time optimization algorithm with linear-quadratic convergence. Preprint, submitted May 9, arXiv:1505.02250.Google Scholar
• Robbins H, Monro S (1951) A stochastic approximation method. Ann. Math. Statist. 22(3):400–407.Google Scholar
• Spall JC (2003) Introduction to Stochastic Search and Optimization (John Wiley & Sons, Inc., Hoboken, NJ).Google Scholar
• Yousefian F, Nedić A, Shanbhag UV (2011) On stochastic gradient and subgradient methods with adaptive steplength sequences. Automatica 48(1):56–67.Google Scholar