Constraint Learning to Define Trust Regions in Optimization over Pre-Trained Predictive Models

References

• Altman NS (1992) An introduction to kernel and nearest-neighbor nonparametric regression. Amer. Statist. 46(3):175–185.Crossref, Google Scholar
• Amodei D, Olah C, Steinhardt J, Christiano P, Schulman J, Mané D (2016) Concrete problems in ai safety. Preprint, submitted June 21, https://arxiv.org/abs/1606.06565.Google Scholar
• Anderson R, Huchette J, Ma W, Tjandraatmadja C, Vielma JP (2020) Strong mixed-integer programming formulations for trained neural networks. Math. Programming 183(1–2):3–39.Crossref, Google Scholar
• Baardman L, Cohen MC, Panchamgam K, Perakis G, Segev D (2019) Scheduling promotion vehicles to boost profits. Management Sci. 65(1):50–70.Link, Google Scholar
• Bergman D, Huang T, Brooks P, Lodi A, Raghunathan AU (2022) Janos: An integrated predictive and prescriptive modeling framework. INFORMS J. Comput. 34(2):807–816.Link, Google Scholar
• Bertsimas D, O’Hair A, Relyea S, Silberholz J (2016) An analytics approach to designing combination chemotherapy regimens for cancer. Management Sci. 62(5):1511–1531.Link, Google Scholar
• Biggs M, Hariss R, Perakis G (2023) Constrained optimization of objective functions determined from random forests. Production Oper. Management 32(2):397–415.Google Scholar
• Biggs M, Sun W, Ettl M (2021) Model distillation for revenue optimization: Interpretable personalized pricing. Meila M, Zhang T, eds. Proc. Internat. Conf. on Machine Learn. (JMLR, Cambridge, MA), 946–956.Google Scholar
• Botoeva E, Kouvaros P, Kronqvist J, Lomuscio A, Misener R (2020) Efficient verification of relu-based neural networks via dependency analysis. Proc. Conf. AAAI Artificial Intelligence 34:3291–3299.Crossref, Google Scholar
• Box GE (1954) Some theorems on quadratic forms applied in the study of analysis of variance problems: I. Effect of inequality of variance in the one-way classification. Ann. Math. Statist. 25(2):290–302.Crossref, Google Scholar
• Bunel RR, Turkaslan I, Torr P, Kohli P, Mudigonda PK (2018) A unified view of piecewise linear neural network verification. Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R, eds. Adv. Neural Inform. Processing Systems (Curran Associates, Red Hook, NY), 31.Google Scholar
• Cheng CH, Nührenberg G, Ruess H (2017) Maximum resilience of artificial neural networks. Proc. Internat. Sympos. on Automated Tech. for Verification and Analysis (Springer, Berlin), 251–268.Google Scholar
• Cortez P, Cerdeira A, Almeida F, Matos T, Reis J (2009) Modeling wine preferences by data mining from physicochemical properties. Decision Support Systems 47(4):547–553.Crossref, Google Scholar
• Dutta S, Jha S, Sankaranarayanan S, Tiwari A (2018) Output range analysis for deep feedforward neural networks. Proc. NASA Formal Methods Sympos. (Springer, Berlin), 121–138.Google Scholar
• Ferreira KJ, Lee BHA, Simchi-Levi D (2016) Analytics for an online retailer: Demand forecasting and price optimization. Manufacturing Service Oper. Management 18(1):69–88.Link, Google Scholar
• Fischetti M, Jo J (2018) Deep neural networks and mixed integer linear optimization. Constraints 23(3):296–309.Crossref, Google Scholar
• Friedman JH (2017) The Elements of Statistical Learning: Data Mining, Inference, and Prediction (Springer, Berlin).Google Scholar
• Gavana A (2022) Index of test functions in global optimization problems. Accessed October 3, 2022, http://infinity77.net/global_optimization/test_functions.html#multidimensional-test-functions-index.Google Scholar
• Grimstad B, Andersson H (2019) Relu networks as surrogate models in mixed-integer linear programs. Comput. Chemical Engrg. 131:106580.Crossref, Google Scholar
• Gurobi Optimization LLC (2020) Gurobi optimizer reference manual. Accessed June 6, 2023, https://www.gurobi.com.Google Scholar
• Hofmann T, Schölkopf B, Smola AJ (2008) Kernel methods in machine learning. Ann. Statist. 36(3):1171–1220.Crossref, Google Scholar
• Huang T, Bergman D, Gopal R (2019) Predictive and prescriptive analytics for location selection of add-on retail products. Production Oper. Management 28(7):1858–1877.Crossref, Google Scholar
• Hubert M, Rousseeuw PJ, Vanden Branden K (2005) Robpca: A new approach to robust principal component analysis. Technometrics 47(1):64–79.Crossref, Google Scholar
• Johnson RA, Wichern DW (2014) Applied Multivariate Statistical Analysis, 6th ed., vol. 6 (Pearson, London).Google Scholar
• Jolliffe IT (2002) Principal Component Analysis for Special Types of Data (Springer, Berlin).Google Scholar
• Katz G, Barrett C, Dill DL, Julian K, Kochenderfer MJ (2017) Reluplex: An efficient smt solver for verifying deep neural networks. Proc. Internat. Conf. on Computer Aided Verification (Springer, Berlin), 97–117.Google Scholar
• Leys C, Klein O, Dominicy Y, Ley C (2018) Detecting multivariate outliers: Use a robust variant of the Mahalanobis distance. J. Experiment. Soc. Psych. 74:150–156.Crossref, Google Scholar
• Liu S, He L, Max Shen ZJ (2021) On-time last-mile delivery: Order assignment with travel-time predictors. Management Sci. 67(7):4095–4119.Link, Google Scholar
• Liu FT, Ting KM, Zhou ZH (2008) Isolation forest. Proc. 8th IEEE Internat. Conf. on Data Mining (IEEE, New York), 413–422.Google Scholar
• Lomuscio A, Maganti L (2017) An approach to reachability analysis for feed-forward relu neural networks. Preprint, submitted June 22, https://arxiv.org/abs/1706.07351.Google Scholar
• Mahalanobis PC (1936) On the Generalized Distance in Statistics (National Institute of Science of India, Delhi).Google Scholar
• Maragno D, Wiberg H, Bertsimas D, Birbil SI, Hertog DD, Fajemisin A (2023) Mixed-integer optimization with constraint learning. Operations Res., ePub ahead of print December 1, https://doi.org/10.1287/opre.2021.0707.Google Scholar
• Mišić VV (2020) Optimization of tree ensembles. Oper. Res. 68(5):1605–1624.Link, Google Scholar
• Mistry M, Letsios D, Krennrich G, Lee RM, Misener R (2021) Mixed-integer convex nonlinear optimization with gradient-boosted trees embedded. INFORMS J. Comput. 33(3):1103–1119.Link, Google Scholar
• Papageorgiou I (2020) Ceramic investigation: How to perform statistical analyses. Archaeology Anthropology Sci. 12(9):1–19.Google Scholar
• Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, et al. (2011) Scikit-learn: Machine learning in Python. J. Machine Learn. Res. 12:2825–2830.Google Scholar
• Sahinidis NV (1996) BARON: A general purpose global optimization software package (Version 23.3.11). J. Global Optim. 8:201–205.Google Scholar
• Schölkopf B, Williamson RC, Smola A, Shawe-Taylor J, Platt J (1999) Support vector method for novelty detection. Solla S, Leen T, Müller K, eds. Adv. Neural Inform. Processing Systems (MIT Press, Cambridge, MA), 12.Google Scholar
• Schweidtmann AM, Mitsos A (2019) Deterministic global optimization with artificial neural networks embedded. J. Optim. Theory Appl. 180(3):925–948.Crossref, Google Scholar
• Schweidtmann AM, Weber JM, Wende C, Netze L, Mitsos A (2022) Obey validity limits of data-driven models through topological data analysis and one-class classification. Optim. Engrg. 23(2):855–876.Crossref, Google Scholar
• Serra T, Tjandraatmadja C, Ramalingam S (2018) Bounding and counting linear regions of deep neural networks. Dy J, Krause A, eds. Proc. Internat. Conf. on Machine Learn. (JMLR, Cambridge, MA), 4558–4566.Google Scholar
• Shi C, Emadikhiav M, Lozano L, Bergman D (2024) Constraint learning to define trust regions in optimization over pre-trained predictive models. https://dx.doi.org/10.1287/ijoc.2022.0312.cd, https://github.com/INFORMSJoC/2022.0312.Google Scholar
• Tax D (2001) One class classification: Concept learning in the absence of counter examples. PhD dissertation, Netherlands Delft University of Technology, Delft.Google Scholar
• Teixeira AP, Clemente JJ, Cunha AE, Carrondo MJ, Oliveira R (2006) Bioprocess iterative batch-to-batch optimization based on hybrid parametric/nonparametric models. Biotech. Progress 22(1):247–258.Crossref, Google Scholar
• The MathWorks Inc (2010) peaks minimization with globalsearch. Accessed October 3, 2022, https://www.mathworks.com/matlabcentral/mlc-downloads/downloads/submissions/27178/versions/6/previews/html/gsPeaksExample.html.Google Scholar
• Tjeng V, Xiao K, Tedrake R (2017) Evaluating robustness of neural networks with mixed integer programming. Preprint, submitted November 20, https://arxiv.org/abs/1711.07356.Google Scholar
• Tsay C, Kronqvist J, Thebelt A, Misener R (2021) Partition-based formulations for mixed-integer optimization of trained relu neural networks. Preprint, submitted February 8, https://arxiv.org/abs/2102.04373.Google Scholar
• Verwer S, Zhang Y, Ye QC (2017) Auction optimization using regression trees and linear models as integer programs. Artificial Intelligence 244:368–395.Crossref, Google Scholar
• Wang K, Lozano L, Bergman D, Cardonha C (2021) A two-stage exact algorithm for optimization of neural network ensemble. Prog. Internat. Conf. Integration Constraint Programming Artificial Intelligence Oper. Res. (Springer, Berlin), 106–114.Google Scholar
• Wang K, Lozano L, Cardonha C, Bergman D (2023) Optimizing over an ensemble of trained neural networks. INFORMS J. Comput. 35(3):652–674.Link, Google Scholar
• Zheng S, Zhu YX, Li DQ, Cao ZJ, Deng QX, Phoon KK (2021) Probabilistic outlier detection for sparse multivariate geotechnical site investigation data using Bayesian learning. Geosci. Frontiers 12(1):425–439.Crossref, Google Scholar