Sample and Computationally Efficient Stochastic Kriging in High Dimensions

References

• Ababou R, Bagtzoglou AC, Wood EF (1994) On the condition number of covariance matrices in kriging, estimation, and simulation of random fields. Math. Geol. 26(1):99–133.Crossref, Google Scholar
• Adler RJ (1981) The Geometry of Random Fields (Wiley, Chichester, UK).Google Scholar
• Ankenman B, Nelson BL, Staum J (2010) Stochastic kriging for simulation metamodeling. Oper. Res. 58(2):371–382.Link, Google Scholar
• Arfken GB, Weber HJ, Harris FE (2012) Mathematical Methods for Physicists: A Comprehensive Guide, 7th ed. (Academic Press, London).Google Scholar
• Barton RR (2015) Tutorial: Simulation metamodeling. Proc. 2015 Winter Simulation Conf., 1765–1779.Google Scholar
• Barton RR, Nelson BL, Xie W (2014) Quantifying input uncertainty via simulation confidence intervals. INFORMS J. Comput. 26(1):74–87.Link, Google Scholar
• Binois M, Gramacy RB, Ludkovski M (2018) Practical heteroscedastic Gaussian process modeling for large simulation experiments. J. Comput. Graph. Statist. 27(4):808–821.Crossref, Google Scholar
• Buchholz P, Thümmler A (2005) Enhancing evolutionary algorithms with statistical selection procedures for simulation optimization. Proc. 2005 Winter Simulation Conf. (Piscataway, NJ), 842–852.Google Scholar
• Bungartz HJ, Griebel M (2004) Sparse grids. Acta Numer. 13:147–269.Crossref, Google Scholar
• Burt DR, Rasmussen CE, van der Wilk M (2020) Convergence of sparse variational inference in Gaussian processes regression. J. Mach. Learn. Res. 21(131):1–63.Google Scholar
• Caponnetto A, De Vito E (2007) Optimal rates for the regularized least-squares algorithm. Found. Comput. Math. 7(3):331–368.Crossref, Google Scholar
• Carnal E, Walsh J (1991) Markov properties for certain random fields. Mayer-Wolf E, Merzbach E, Shwartz A, eds., Stochastic Analysis: Liber Amicorum for Moshe Zakai (Academic Press, London), 91–110.Crossref, Google Scholar
• Dolph CL, Woodbury MA (1952) On the relation between Green’s functions and covariances of certain stochastic processes and its application to unbiased linear prediction. Trans. Amer. Math. Soc. 72(3):519–550.Google Scholar
• Evans LC (2010) Partial Differential Equations, 2nd ed. (American Mathematical Society, Providence, RI).Crossref, Google Scholar
• Fanuel M, Schreurs J, Suykens J (2021) Diversity sampling is an implicit regularization for kernel methods. SIAM J. Math. Data Sci. 3(1):280–297.Crossref, Google Scholar
• Garcke J, Griebel M, eds. (2013) Sparse Grids and Applications (Springer-Verlag, Berlin).Crossref, Google Scholar
• Gramacy RB (2020) Surrogates: Gaussian Process Modeling, Design and Optimization for the Applied Sciences (CRC Press, Boca Raton, FL).Crossref, Google Scholar
• Gramacy RB, Apley DW (2015) Local Gaussian process approximation for large computer experiments. J. Comput. Graph. Statist. 24(2):561–578.Crossref, Google Scholar
• Haaland B, Qian PZG (2011) Accurate emulators for large-scale computer experiments. Ann. Statist. 39(6):2974–3002.Crossref, Google Scholar
• Haaland B, Wang W, Maheshwari V (2018) A framework for controlling sources of inaccuracy in Gaussian process emulation of deterministic computer experiments. SIAM/ASA J. Uncertain. Quantif. 6(2):497–521.Crossref, Google Scholar
• Horn RA, Johnson CR (2012) Matrix Analysis, 2nd ed. (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• Jacot A, Şimşek B, Spadaro F, Hongler C, Gabriel F (2020) Implicit regularization of random feature models. Proceedings of the 37th International Conference on Machine Learning (JMLR.org), 4631–4640.Google Scholar
• Jalali H, Van Nieuwenhuyse I, Picheny V (2017) Comparison of kriging-based algorithms for simulation optimization with heterogeneous noise. Eur. J. Oper. Res. 261(1):279–301.Crossref, Google Scholar
• Karatzas I, Shreve SE (1991) Brownian Motion and Stochastic Calculus, 2nd ed. (Springer, New York).Google Scholar
• Kleijnen JPC (2017) Regression and kriging metamodels with their experimental designs in simulation: A review. Eur. J. Oper. Res. 256(1):1–16.Crossref, Google Scholar
• Liu F, Huang X, Chen Y, Suykens JA (2022) Random features for kernel approximation: A survey on algorithms, theory, and beyond. IEEE Trans. Pattern Anal. Mach. Intell. 44(10):7128–7148.Google Scholar
• Liu H, Ong YS, Shen X, Cai J (2020) When Gaussian process meets big data: A review of scalable GPs. IEEE Trans. Neural Netw. Learn. Syst. 31(11):4405–4423.Crossref, Google Scholar
• Lu X, Rudi A, Borgonovo E, Rosasco L (2020) Faster kriging: Facing high-dimensional simulators. Oper. Res. 68(1):233–249.Link, Google Scholar
• Marcus MB, Rosen J (2006) Markov Processes, Gaussian Processes, and Local Times (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• Matheron G (1963) Principles of geostatistics. Econ. Geol. 58(8):1246–1266.Crossref, Google Scholar
• McKay MD, Beckman RJ, Conover WJ (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics. 21(2):239–245.Google Scholar
• Moura JMF, Goswami S (1997) Gauss–Markov random fields (GMrf) with continuous indices. IEEE Trans. Inform. Theory. 43(5):1560–1573.Crossref, Google Scholar
• O’Hagan A (1991) Bayes–Hermite quadrature. J. Statist. Plann. Inference 29(3):245–260.Crossref, Google Scholar
• Pati D, Bhattacharya A, Cheng G (2015) Optimal Bayesian estimation in random covariate design with a rescaled Gaussian process prior. J. Mach. Learn. Res. 16(87):2837–2851.Google Scholar
• Pearce MAL, Poloczek M, Branke J (2022) Bayesian optimization allowing for common random numbers. Oper. Res. Forthcoming.Link, Google Scholar
• Plumlee M (2014) Fast prediction of deterministic functions using sparse grid experimental designs. J. Amer. Statist. Assoc. 109(508):1581–1591.Crossref, Google Scholar
• Rahimi A, Recht B (2007) Random features for large-scale kernel machines. Adv. Neural Inf. Process. Syst. 20:1177–1184.Google Scholar
• Rasmussen CE, Williams KI (2006) Gaussian Processes for Machine Learning (MIT Press, Cambridge, MA).Google Scholar
• Rudi A, Rosasco L (2017) Generalization properties of learning with random features. Adv. Neural Inf. Process. Syst. 30:3218–3228.Google Scholar
• Rue H, Held L (2005) Gaussian Markov Random Fields: Theory and Applications (CRC Press, Boca Raton, FL).Crossref, Google Scholar
• Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Statist. Sci. 4(4):409–423.Crossref, Google Scholar
• Salemi P, Staum J, Nelson BL (2019a) Generalized integrated Brownian fields for simulation metamodeling. Oper. Res. 67(3):874–891.Link, Google Scholar
• Salemi PL, Song E, Nelson BL, Staum J (2019b) Gaussian Markov random fields for discrete optimization via simulation: Framework and algorithms. Oper. Res. 67(1):250–266.Link, Google Scholar
• Santner TJ, Williams BJ, Notz WI (2003) The Design and Analysis of Computer Experiments, 2nd ed. (Springer, New York).Crossref, Google Scholar
• Schölkopf B, Smola AJ (2002) Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (MIT Press, Cambridge, MA).Google Scholar
• Shahriari B, Swersky K, Wang Z, Adams RP, de Freitas N (2016) Taking the human out of the loop: A review of Bayesian optimization. Proc. IEEE, 104(1):148–175.Crossref, Google Scholar
• Smola AJ, Schölkopf B (2000) Sparse greedy matrix approximation for mahine learning. Proceedings of the 17th International Conference on Machine Learning (Morgan Kaufmann Publishers, San Francisco), 911–918.Google Scholar
• Stone CJ (1980) Optimal rates of convergence for nonparametric estimators. Ann. Statist. 8(6):1348–1360.Crossref, Google Scholar
• Sun L, Hong LJ, Hu Z (2014) Balancing exploitation and exploration in discrete optimization via simulation through a Gaussian process-based search. Oper. Res. 62(6):1416–1438.Link, Google Scholar
• Tsybakov AB (2009) Introduction to Nonparametric Estimation (Springer Science+Business Media, New York).Crossref, Google Scholar
• Tuo R, Wang W (2020) Kriging prediction with isotropic Matérn correlations: Robustness and experimental design. J. Mach. Learn. Res. 21(187):1–38.Google Scholar
• Tuo R, Wang Y, Wu CFJ (2020) On the improved rates of convergence for Matérn-type kernel ridge regression with application to calibration of computer models. SIAM/ASA J. Uncertain. Quantif. 8(4):1522–1547.Crossref, Google Scholar
• van Dam ER, Husslage B, den Hertog D, Melissen H (2007) Maximin Latin hypercube designs in two dimensions. Oper. Res. 55(1):158–169.Link, Google Scholar
• van der Vaart A, van Zanten H (2011) Information rates of nonparametric Gaussian process methods. J. Mach. Learn. Res. 12(2011):2095–2119.Google Scholar
• Wang W, Haaland B (2019) Controlling sources of inaccuracy in stochastic kriging. Technometrics. 61(3):309–321.Crossref, Google Scholar
• Wang W, Tuo R, Wu CFJ (2020) On prediction properties of kriging: Uniform error bounds and robustness. J. Amer. Statist. Assoc. 115(530):920–930.Crossref, Google Scholar
• Williams C, Seeger M (2000) Using the Nyström method to speed up kernel machines. Adv. Neural Inf. Process. Syst. 13:682–688.Google Scholar
• Xie W, Nelson BL, Barton RR (2014) A Bayesian framework for quantifying uncertainty in stochastic simulation. Oper. Res. 62(6):1439–1452.Link, Google Scholar
• Ylvisaker D (1987) Prediction and design. Ann. Statist. 15(1):1–19.Crossref, Google Scholar
• Zenger C (1991) Sparse grids. Hackbusch W, ed., Parallel Algorithms for Partial Differential Equations, volume 31 of Notes on Numerical Fluid Mechanics (Vieweg-Verlag, Braunschweig, Germany), 241–251 (Vieweg-Verlag, Braunschweig).Google Scholar
• Zhang B, Cole DA, Gramacy RB (2021) Distance-distributed design for Gaussian process surrogates. Technometrics. 63(1):40–52.Crossref, Google Scholar