Parallel Bayesian Global Optimization of Expensive Functions

References

• Ababou R, Bagtzoglou AC, Wood EF (1994) On the condition number of covariance matrices in kriging, estimation, and simulation of random fields. Math. Geology 26(1):99–133.Crossref, Google Scholar
• Amatriain X (2014) 10 Lessons Learned From Building ML Systems. Recording of presentation from MLconf 2014. Accessed November 26, 2015, https://www.youtube.com/watch?v=WdzWPuazLA8.Google Scholar
• Boender CGE, Kan AR (1987) Bayesian stopping rules for multistart global optimization methods. Math. Programming 37(1):59–80.Crossref, Google Scholar
• Brochu E, Brochu T, de Freitas N (2010) A Bayesian interactive optimization approach to procedural animation design. Proc. 2010 ACM SIGGRAPH/Eurographics Sympos. Comput. Animation (Eurographics Association, Madrid), 103–112.Google Scholar
• Calvin JM (1997) Average performance of a class of adaptive algorithms for global optimization. Ann. Appl. Probab. 7(3):711–730.Crossref, Google Scholar
• Calvin JM, Žilinskas A (2002) One-dimensional global optimization based on statistical models. Dzemyda G, Šaltenis V, Žilinskas A, eds. Stochastic and Global Optimization (Springer, New York), 49–63.Crossref, Google Scholar
• Calvin J, Žilinskas A (2005) One-dimensional global optimization for observations with noise. Comput. Math. Appl. 50(1):157–169.Crossref, Google Scholar
• Chevalier C, Ginsbourger D (2013) Fast computation of the multi-points expected improvement with applications in batch selection. Internat. Conf. Learning Intelligent Optim. (Springer, New York), 59–69.Google Scholar
• Clark S (2014) Introducing “MOE”: metric optimization engine; a new open source, machine learning service for optimal experiment design. Accessed November 26, 2015, http://engineeringblog.yelp.com/2014/07/introducing-moe-metric-optimization-engine-a-new-open-source-machine-learning-service-for-optimal-ex.html.Google Scholar
• Clark SC, Liu E, Frazier PI, Wang J, Oktay D, Vesdapunt N (2014) Metrics optimization engine. Accessed September 17, 2017, http://yelp.github.io/MOE/.Google Scholar
• Dennis JE Jr, Torczon V (1991) Direct search methods on parallel machines. SIAM J. Optim. 1(4):448–474.Crossref, Google Scholar
• Frazier P, Powell W, Dayanik S (2009) The knowledge-gradient policy for correlated normal beliefs. INFORMS J. Comput. 21(4):599–613.Link, Google Scholar
• Frazier PI, Wang J (2016) Bayesian optimization for materials design. Lookman T, Alexander FJ, Rajan K, eds. Information Science for Materials Discovery and Design (Springer, New York), 45–75.Crossref, Google Scholar
• Frazier PI, Xie J, Chick SE (2011) Value of information methods for pairwise sampling with correlations. Proc. Winter Simulation Conf. (IEEE, Washington DC), 3979–3991.Google Scholar
• Genz A (1992) Numerical computation of multivariate normal probabilities. J. Comput. Graphics Statist. 1(2):141–149.Google Scholar
• Ginsbourger D (2009) Two advances in Gaussian process-based prediction and optimization for computer experiments. Report, MASCOT09 Meeting, MASCOT09, Villetaneuse, France.Google Scholar
• Ginsbourger D, Le Riche R, Carraro L (2008) A multi-points criterion for deterministic parallel global optimization based on Gaussian processes. Working Paper hal-00260579, Ecole Nationale Superieure des Mines, Sainte-Etienne, France.Google Scholar
• Ginsbourger D, Le Riche R, Carraro L (2010) Kriging is well-suited to parallelize optimization. Tenne Y, Goh CK, eds. Computational Intelligence in Expensive Optimization Problems (Springer, New York), 131–162.Crossref, Google Scholar
• Ginsbourger D, Picheny V, Roustant O, Binois M, Chevalier C, Marmin S, Wagner T (2015) Diceoptim: Kriging-based optimization for computer experiments. Accessed February 13, 2016, https://cran.r-project.org/web/packages/DiceOptim/index.html.Google Scholar
• Ho YC (1987) Performance evaluation and perturbation analysis of discrete event dynamic systems. IEEE Trans. Automat. Control. 32(7):563–572.Crossref, Google Scholar
• Holland JH (1992) Adaptation in Natural and Artificial Systems: an Introductory Analysis With Applications to Biology, Control, and Artificial Intelligence (MIT Press, Cambridge, MA).Crossref, Google Scholar
• Howard RA (1966) Information value theory. IEEE Trans. Systems Sci. Cybernetics 2(1):22–26.Crossref, Google Scholar
• Huang D, Allen TT, Notz WI, Zeng N (2006) Global optimization of stochastic black-box systems via sequential kriging meta-models. J. Global Optim. 34(3):441–466.Crossref, Google Scholar
• Jamil M, Yang XS (2013) A literature survey of benchmark functions for global optimisation problems. Internat. J. Math. Model. Numer. Optim. 4(2):150–194.Google Scholar
• Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J. Global Optim. 13(4):455–492.Crossref, Google Scholar
• Kennedy J (2010) Particle Swarm Optimization. Sammut C, Webb GI, eds. Encyclopedia of Machine Learning (Springer, New York), 760–766.Google Scholar
• Kim SH, Nelson BL (2007) Recent advances in ranking and selection. Proc. 39th Conf. Winter Simulation (IEEE Press, Washington, DC), 162–172.Google Scholar
• Kushner H, Yin GG (2003) Stochastic Approximation and Recursive Algorithms and Applications, vol. 35 (Springer Science & Business Media, Berlin).Google Scholar
• Kushner HJ (1964) A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. J. Fluids Engrg. 86(1):97–106.Google Scholar
• Liu DC, Nocedal J (1989) On the limited memory BFGS method for large scale optimization. Math. Programming 45(1–3):503–528.Crossref, Google Scholar
• Marmin S, Chevalier C, Ginsbourger D (2015) Differentiating the multipoint expected improvement for optimal batch design. Pardalos P, Pavone M, Farinella GM, Cutello V, eds. Machine Learning, Optimization, and Big Data (Springer, New York), 37–48.Crossref, Google Scholar
• McKay MD, Beckman RJ, Conover WJ (2000) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 42(1):55–61.Crossref, Google Scholar
• Mockus J (1989) The Bayesian Approach to Local Optimization (Springer, New York).Crossref, Google Scholar
• Mockus J, Tiesis V, Zilinskas A (1978) The application of Bayesian methods for seeking the extremum. Dixon LCW, Szego GP, eds. Toward Global Optimization, vol. 2 (North-Holland, Amsterdam), 117–129.Google Scholar
• Polyak BT (1990) New stochastic approximation type procedures. Automat. i Telemekh 7(2):98–107.Google Scholar
• Rasmussen C, Williams C (2006) Gaussian Processes for Machine Learning (MIT Press, Cambridge, MA). Accessed January 29, 2020, http://www.gaussianprocess.org/gpml.Google Scholar
• Ruppert D (1988) Efficient estimations from a slowly convergent Robbins-Monro process. Technical Report, Cornell University Operations Research and Industrial Engineering, Ithaca, NY.Google Scholar
• Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Statist. Sci. 4(4):409–423.Crossref, Google Scholar
• Smith SP (1995) Differentiation of the Cholesky algorithm. J. Comput. Graph. Statist. 4(2):134–147.Google Scholar
• Snoek J, Larochelle H, Adams RP (2012) Practical Bayesian optimization of machine learning algorithms. Proc. 25th Internat. Conf. Neural Inform. Processing Systems, vol. 2 (Curran Associates Inc., Red Hook, NY), 2951–2959.Google Scholar
• Vazquez E, Bect J (2010) Convergence properties of the expected improvement algorithm with fixed mean and covariance functions. J. Statist. Plann. Inference 140(11):3088–3095.Crossref, Google Scholar
• Virtanen P, Gommers R, Oliphant TE, Haberland M, Reddy T, Cournapeau D, Burovski E, et al.. (2019) SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python. Preprint submitted July, https://ui.adsabs.harvard.edu/abs/2019arXiv190710121V.Google Scholar
• Villemonteix J, Vazquez E, Walter E (2009) An informational approach to the global optimization of expensive-to-evaluate functions. J. Global Optim. 44(4):509–534.Crossref, Google Scholar
• Xie J, Frazier PI, Chick SE (2016) Bayesian optimization via simulation with pairwise sampling and correlated prior beliefs. Oper. Res. 64(2):542–559.Link, Google Scholar