Actor-Critic–Like Stochastic Adaptive Search for Continuous Simulation Optimization

References

• Andradóttir S, Prudius AA (2009) Balanced explorative and exploitative search with estimation for simulation optimization. INFORMS J. Comput. 21(2):193–208.Link, Google Scholar
• Andradóttir S, Prudius AA (2010) Adaptive random search for continuous simulation optimization. Naval Res. Logist. 57(6):583–604.Crossref, Google Scholar
• Ankenman B, Nelson BL, Staum J (2010) Stochastic kriging for simulation metamodeling. Oper. Res. 58(2):371–382.Link, Google Scholar
• Ball R, Branke J, Meisel S (2018) Optimal sampling for simulated annealing under noise. INFORMS J. Comput. 30(1):200–215.Link, Google Scholar
• Barton RR (2009) Simulation optimization using metamodels. Proc. 2009 Winter Simulation Conf. (IEEE, New York), 230–238.Google Scholar
• Barton RR, Meckesheimer M (2006) Metamodel-based simulation optimization. Handbook Oper. Res. Management Sci. 13:535–574.Google Scholar
• Baumert S, Smith RL (2002) Pure random search for noisy objective functions. Technical report 01-03, University of Michigan, Ann Arbor.Google Scholar
• Bertsekas DP, Tsitsiklis JN (1996) Neuro-Dynamic Programming (Athena Scientific, Belmont, MA).Google Scholar
• Bishop CM (1995) Neural Networks for Pattern Recognition (Oxford University Press, New York).Crossref, Google Scholar
• Das S, Suganthan PN (2010) Problem definitions and evaluation criteria for cec 2011 competition on testing evolutionary algorithms on real world optimization problems. Technical report, Jadavpur University, Kolkata, India and Nanyang Technological University, Singapore.Google Scholar
• Dorigo M, Blum C (2005) Ant colony optimization theory: A survey. Theoretical Comput. Sci. 344(2-3):243–278.Crossref, Google Scholar
• Fan Q, Hu J (2018) Surrogate-based promising area search for Lipschitz continuous simulation optimization. INFORMS J. Comput. 30(4):677–693.Link, Google Scholar
• Glasserman P (2003) Monte Carlo Methods in Financial Engineering (Springer Science & Business Media, New York).Crossref, Google Scholar
• Gosavi A (2009) Reinforcement learning: A tutorial survey and recent advances. INFORMS J. Comput. 21(2):178–192.Link, Google Scholar
• Gutmann HM (2001) A radial basis function method for global optimization. J. Global Optim. 19:201–227.Crossref, Google Scholar
• Hajek B (1998) Cooling schedules for optimal annealing. Math. Oper. Res. 13(2):311–329.Link, Google Scholar
• Hansen N, Ostermeier A (1996) Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation. Proc IEEE Internat. Conf. on Evolutionary Comput. (IEEE, New York), 312–317.Google Scholar
• He D, Lee LH, Chen CH, Fu MC, Wasserkrug S (2010) Simulation optimization using the cross-entropy method with optimal computing budget allocation. ACM Trans. Modeling Comput. Simulation 20(1):1–22.Crossref, Google Scholar
• Hong LJ, Nelson BL (2006) Discrete optimization via simulation using compass. Oper. Res. 54(1):115–129.Link, Google Scholar
• Hu J (2015) Model-based stochastic search methods. Fu MC, ed., Handbook of Simulation Optimization (Springer, New York), 319–340.Crossref, Google Scholar
• Hu J, Hu P (2011) Annealing adaptive search, cross-entropy, and stochastic approximation in global optimization. Naval Res. Logist. 58(5):457–477.Crossref, Google Scholar
• Hu J, Zhou E, Fan Q (2014) Model-based annealing random search with stochastic averaging. ACM Trans. Modeling Comput. Simulation 24(4):1–23.Crossref, Google Scholar
• Hu J, Chang HS, Fu MC, Marcus SI (2011) Dynamic sample budget allocation in model-based optimization. J. Global Optim. 50:575–596.Crossref, Google Scholar
• Huang D, Allen TT, Notz WI, Zeng N (2006) Global optimization of stochastic black-box systems via sequential kriging meta-models. J. Global Optim. 34(3):441–466.Crossref, Google Scholar
• Jalali H, Nieuwenhuyse IV, Picheny V (2017) Comparison of kriging-based algorithms for simulation optimization with heterogeneous noise. Eur. J. Oper. Res. 261(1):279–301.Crossref, Google Scholar
• Jones D, Schonlau M, Welch W (1998) Efficient global optimization of expensive black-box functions. J. Global Optim. 13:455–492.Crossref, Google Scholar
• Kaelbing L, Littman M, Moore A (1996) Reinforcement learning: A survey. J. Artificial Intelligence Res. 4:237–285.Crossref, Google Scholar
• Kiatsupaibul S, Smith RL, Zabinsky ZB (2018) Single observation adaptive search for continuous simulation optimization. Oper. Res. 66(6):1713–1727.Link, Google Scholar
• Kiefer J, Wolfowitz J (1952) Stochastic estimation of the maximum of a regression function. Ann. Math. Statist. 23(3):462–466.Crossref, Google Scholar
• Kim S, Pasupathy R, Henderson SG (2015) A guide to sample average approximiation. Fu MC, ed. Handbook of Simulation Optimization (Springer, New York), 207–243.Crossref, Google Scholar
• Kleijnen JPC (2014) Simulation-optimization via kriging and bootstrapping: a survey. J. Simul. 8(4):241–250.Crossref, Google Scholar
• Kleijnen JPC (2015) Response surface methodology. Fu MC, ed. Handbook of Simulation Optimization (Springer, New York), 81–104.Crossref, Google Scholar
• Kleywegt A, Shapiro A, Homem-De-Mello T (2001) The sample average approximation method for stochastic discrete optimization. SIAM J. Optim. 12(2):479–502.Crossref, Google Scholar
• Konda VR, Borkar VS (1999) Actor-critic-type learning algorithms for Markov decision processes. SIAM J. Control Optim. 38(1):94–123.Crossref, Google Scholar
• Konda VR, Tsitsiklis JN (2003) On actor-critic algorithms. SIAM J. Control Optim. 42(4):1143–1166.Crossref, Google Scholar
• Kushner HJ, Clark DS (1978) Stochastic Approximation Methods for Constrained and Unconstrained Systems (Springer-Verlag, New York).Crossref, Google Scholar
• Larrãnaga P, Lozano J, eds. (2002) Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. (Kluwer Academic Publisher, Boston, MA).Crossref, Google Scholar
• Loeppky JL, Sacks J, Welch WJ (2009) Choosing the sample size of a computer experiment: A practical guide. Technometrics 51(4):366–376.Crossref, Google Scholar
• Picheny V, Wagner T, Ginsbourger D (2013) A benchmark of kriging-based infill criteria for noisy optimization. Structural Multidisciplinary Optim. 48(3):607–626.Crossref, Google Scholar
• Prudius A, Andradóttir S (2012) Averaging frameworks for simulation optimization with applications to simulated annealing. Naval Res. Logist. 59(6):411–429.Crossref, Google Scholar
• Robbins H, Monro S (1951) A stochastic approximation method. Ann. Math. Statist. 22(3):400–407.Crossref, Google Scholar
• Rubinstein RY, Kroese DP (2004) The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation (Information Science and Statistics) (Springer-Verlag, New York).Crossref, Google Scholar
• Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Statist. Sci. 4(4):409–423.Crossref, Google Scholar
• Shi L, Ólafsson S (2000) Nested partitions method for stochastic optimization. Methodology Comput. Appl. Probability 2(3):271–291.Crossref, Google Scholar
• Spall JC (1992) Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Automat. Control 37(3):332–341.Crossref, Google Scholar
• Spall JC (2003) Introduction to Stochastic Search and Optimization (John Wiley & Sons, Hoboken, NJ).Crossref, Google Scholar
• Staum J (2009) Better simulation metamodeling: The why, what, and how of stochastic kriging. Proc. Winter Simulation Conf. (IEEE, New York), 119–133.Google Scholar
• Sutton RS, McAllester D, Singh S, Mansour Y (2000) Policy Gradient Methods for Reinforcement Learning with Function Approximation, vol. 1057–1063. Advances in Neural Information Processing Systems. MIT Press, Cambridge, MA).Google Scholar
• Valdez S, Hernández A, Botello S (2013) A Boltzmann based estimation of distribution algorithm. Inform. Sci. 236(1):126–137.Crossref, Google Scholar
• Wang H, Pasupathy R, Schmeiser BW (2013) Integer-ordered simulation optimization using r-spline: Retrospective search using piecewise-linear interpolation and neighborhood enumeration. ACM Trans. Modeling Comput. Simulation 23(3):1–24.Crossref, Google Scholar
• Williams RJ (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learn. 8:229–256.Crossref, Google Scholar
• Wolpert DH (2004) Finding bounded rational equilibria part I: Iterative focusing. Vincent T, ed. Proc. 11th Internat. Sympos. on Dynamic Games and Appl., Tucson, Arizona. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.154.5730&rep=rep1&type=pdf.Google Scholar
• Xu J, Nelson BL, Hong LJ (2013) An adaptive hyperbox algorithm for high-dimensional discrete optimization via simulation problems. INFORMS J. Comput. 25(1):133–146.Link, Google Scholar
• Yakowitz S, L’Ecuyer P, Vázquez-Abad F (2000) Global stochastic optimization with low-dispersion point sets. Oper. Res. 48(6):939–950.Link, Google Scholar
• Yan D, Mukai H (1992) Stochastic discrete optimization. SIAM J. Control Optim. 30(3):594–612.Crossref, Google Scholar
• Zhang Q, Hu J (2019) Enhancing random search with surrogate models for Lipschitz continuous optimization. Proc. IEEE 15th Internat. Conf. on Automation Sci. and Engrg. (IEEE, New York), 768–773.Google Scholar
• Zlochin M, Birattari M, Meuleau N, Dorigo M (2004) Model-based search for combinatorial optimization: A critical survey. Ann. Oper. Res. 131(1):373–395.Crossref, Google Scholar