Open Problem—Iterative Schemes for Stochastic Optimization: Convergence Statements and Limit Theorems

Central limit theorems represent among the most celebrated of limit theorems in probability theory (Lindeberg 1922, Feller 1945). It may be recalled that the sum of n independent and identically distributed zero mean square integrable random variables grows at the rate of $\sqrt{n}$. Consequently, by dividing this sum by $\sqrt{n}$, we expect its law to possibly approach a limit. This intuition is formalized by the central limit theorem (CLT; Chow and Teicher 2012, chapter 9), and in fact, this limit is the normal distribution. Our interest lies in developing limiting statements for estimators to the following stochastic optimization problem: