Convergence of a Distributed Kiefer-Wolfowitz Algorithm

References

• Kennedy RKL, Khoshgoftaar TM, Villanustre F, Humphrey T (2019) A parallel and distributed stochastic gradient descent implementation using commodity clusters. J. Big Data 6(1):1–23.Google Scholar
• Kiefer J, Wolfowitz J (1952) Stochastic estimation of the maximum of a regression function. Ann. Math. Statist. 23(3):462–466.Google Scholar
• Kushner H, Clark DJ (1978) Stochastic Approximation Methods for Constrained and Unconstrained Systems (Springer, New York).Google Scholar
• Ljung L, Söderström T (1983) Theory and Practice of Recursive Identification (MIT Press, Cambridge, MA).Google Scholar
• Nedic A, Ozdaglar A (2009) Distributed subgradient methods for multi-agent optimization. IEEE Trans. Automatic Control 54(1):48–61.Google Scholar
• Ramaswamy A (2019) DSPG: Decentralized simultaneous perturbations gradient descent scheme. Preprint, submitted March 17, https://arxiv.org/abs/1903.07050.Google Scholar
• Spall J (1992) Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Automatic Control 37(3):332–341.Google Scholar
• Swenson B, Murray R, Kar S, Poor V (2020) Distributed stochastic gradient descent: Nonconvexity, nonsmoothness, and convergence to local minima. Preprint, submitted March 5, https://arxiv.org/abs/2003.02818.Google Scholar