Weighted Optimal Classification Forests

References

• Aghaei S, Gómez A, Vayanos P (2024) Strong optimal classification trees. Oper. Res. 73(4):2223–2241.Google Scholar
• Aglin G, Nijssen S, Schaus P (2021) Assessing optimal forests of decision trees. Proc. IEEE 33rd Internat. Conf. Tools Artificial Intelligence (IEEE Computer Society, Washington, DC), 32–39. Google Scholar
• Akchen YC, Mišić VV (2025) Column-randomized linear programs: Performance guarantees and applications. Oper. Res. 73(3):1366–1383.Link, Google Scholar
• Amorosi L, Padellini T, Puerto J, Valverde C (2024) A mathematical programming approach to sparse canonical correlation analysis. Expert Systems Appl. 237:121293. Crossref, Google Scholar
• Antonio N, de Almeida A, Nunes L (2019) Hotel booking demand data sets. Data Brief 22:41–49.Crossref, Google Scholar
• Baldomero-Naranjo M, Martínez-Merino L, Rodríguez-Chía A (2020) Tightening big MS in integer programming formulations for support vector machines with ramp loss. Eur. J. Oper. Res. 286(1):84–100.Crossref, Google Scholar
• Baldomero-Naranjo M, Martínez-Merino L, Rodríguez-Chía A (2021) A robust SVM-based approach with feature selection and outliers detection for classification problems. Expert Systems Appl. 178:115017.Crossref, Google Scholar
• Benítez S, Blanquero R, Carrizosa E (2019) Cost-sensitive feature selection for support vector machines. Comput. Oper. Res. 106:169–178.Crossref, Google Scholar
• Bernard S, Heutte L, Adam S (2009) On the selection of decision trees in random forests. Proc. Internat. Joint Conf. Neural Networks (IEEE, Piscataway, NJ), 302–307.Google Scholar
• Bertsimas D, Dunn J (2017) Optimal classification trees. Mach. Learn. 106(7):1039–1082.Crossref, Google Scholar
• Bertsimas D, Dunn J, Mundru N (2019b) Optimal prescriptive trees. INFORMS J. Optim. 1(2):164–183.Link, Google Scholar
• Bertsimas D, Dunn J, Paskov I (2022) Stable classification. J. Machine Learn. Res. 23(296):1–53.Google Scholar
• Bertsimas D, Chang A, Mišić VV, Mundru N (2019a) The airlift planning problem. Transportation Sci. 53(3):773–795.Link, Google Scholar
• Biau G, Scornet E (2016) A random forest guided tour. Test 25(2):197–227.Crossref, Google Scholar
• Blanco V, Japón A, Puerto J (2020a) Optimal arrangements of hyperplanes for SVM-based multiclass classification. Adv. Data Anal. Classifications 14(1):175–199.Crossref, Google Scholar
• Blanco V, Japón A, Puerto J (2022) Robust optimal classification trees under noisy labels. Adv. Data Anal. Classifications 16(1):155–179.Crossref, Google Scholar
• Blanco V, Japón A, Puerto J (2023) Multiclass optimal classification trees with SVM-splits. Machine Learn. 112(12):4905–4928.Crossref, Google Scholar
• Blanco V, Puerto J, Rodriguez-Chia AM (2020b) On ℓp-support vector machines and multidimensional kernels. J. Machine Learn. Res. 21(1):469–497.Google Scholar
• Blanco V, Japón A, Puerto J, Zhang P (2026) Weighted optimal classification forests. https://doi.org/10.1287/ijoc.2025.1475.cd, https://github.com/INFORMSJoC/2025.1475.Google Scholar
• Blanquero R, Carrizosa E, Molero C (2021) Optimal randomized classification trees. Comput. Oper. Res. 132:105281.Crossref, Google Scholar
• Breiman L (2001) Random forests. Machine Learn. 45(1):5–32.Crossref, Google Scholar
• Breiman L, Friedman JH, Olshen RA, Stone CG (1984) Classification and Regression Trees (Wadsworth International Group, Belmont, CA).Google Scholar
• Carreira-Perpiñán M, Tavallali P (2018) Alternating optimization of decision trees, with application to learning sparse oblique trees. Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R, eds. Proc. 32nd Internat. Conf. Neural Inform. Processing Systems (Curran Associates, Red Hook, NY), 1219–1229.Google Scholar
• Carrizosa E, Molero C, Romero D (2021) Mathematical optimization in classification and regression trees. TOP 29(1):5–33.Crossref, Google Scholar
• Chen T, Guestrin C (2016) XGBoost: A scalable tree boosting system. Krishnapuram B, Shah M, Smola AJ, Aggarwal C, Shen D, Rastogi R, eds. Proc. 22nd ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (Association for Computing Machinery, New York), 785–794.Google Scholar
• Chen YC, Mišić VV (2022) Decision forest: A nonparametric approach to modeling irrational choice. Management Sci. 68(10):7090–7111.Link, Google Scholar
• Cortes C, Vapnik V (1995) Support-vector networks. Machine Learn. 20(3):273–297.Crossref, Google Scholar
• Demiriz A, Bennett KP, Shawe-Taylor J (2002) Linear programming boosting via column generation. Machine Learn. 46(1):225–254.Crossref, Google Scholar
• Demirović E, Lukina A, Hebrard E (2022) Murtree: Optimal decision trees via dynamic programming and search. J. Machine Learn. Res. 23(26):1–47.Google Scholar
• Eitrich T, Lang B (2006) Efficient optimization of support vector machine learning parameters for unbalanced data sets. J. Comput. Appl. Math. 196(2):425–436.Crossref, Google Scholar
• Firat M, Crognier G, Gabor A (2020) Column generation based heuristic for learning classification trees. Comput. Oper. Res. 116:104866.Crossref, Google Scholar
• Gan J, Li J, Xie Y (2022) Robust SVM for cost-sensitive learning. Neural Processing Lett. 54:2737–2758. Crossref, Google Scholar
• Gnlük O, Kalagnanam J, Li M, Menickelly M, Scheinberg K (2021) Optimal decision trees for categorical data via integer programming. J. Global Optim. 81:233–260.Crossref, Google Scholar
• Grinsztajn L, Oyallon E, Varoquaux G (2022) Why do tree-based models still outperform deep learning on typical tabular data? Proc. 36th Internat. Conf. Neural Inform. Processing Systems (NIPS ’22) (Curran Associates Inc., Red Hook, NY), 507–520.Google Scholar
• Gurobi (2025) Symmetry breaking algorithm in Gurobi. Accessed August 7, 2025, https://support.gurobi.com/hc/en-us/community/posts/360050295511-Symmetry-Breaking-Algorithm-in-Gurobi.Google Scholar
• Hu X, Rudin C, Seltzer M (2019) Optimal sparse decision trees. Wallach HM, Larochelle H, Beygelzimer A, d’Alché-Buc F, Fox EA, Garnett R, eds. Advances in Neural Information Processing Systems, vol. 32 (Curran Associates, Red Hook, NY), 7265–7273.Google Scholar
• Hu H, Siala M, Hebrard E, Huguet M-J (2020) Learning optimal decision trees with MaxSAT and its integration in AdaBoost. Bessiere C, ed. Proc. Twenty-Ninth Internat. Joint Conf. Artificial Intelligence (IJCAI.org), 1170–1176.Google Scholar
• Jo N, Aghaei S, Gómez A, Vayanos P (2023) Learning optimal fair decision trees: Trade-offs between interpretability, fairness, and accuracy. Proc. 2023 AAAI/ACM Conf. AI Ethics Society (AIES ’23) (Association for Computing Machinery, New York), 181–192.Google Scholar
• Kaibel V, Peinhardt M, Pfetsch ME (2011) Orbitopal fixing. Discrete Optim. 8(4):595–610.Crossref, Google Scholar
• Komusiewicz C, Kunz P, Sommer F, Sorge M (2023) On computing optimal tree ensembles. Krause A, Brunskill E, Cho K, Engelhardt B, Sabato S, Scarlett J, eds. Proc. 40th Internat. Conf. Machine Learn., vol. 202 (PMLR, New York), 17364–17374.Google Scholar
• Lichman M (2013) UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences, Irvine.Google Scholar
• Lin J, Zhong C, Hu D, Rudin C, Seltzer M (2020) Generalized and scalable optimal sparse decision trees. Daumé H III, Singh A, eds. Proc. 37th Internat. Conf. Machine Learn., vol. 119 (PMLR, New York), 6150–6160.Google Scholar
• Liu Y (2022) bsnsing: A decision tree induction method based on recursive optimal Boolean rule composition. INFORMS J. Comput. 34(6):2908–2929.Link, Google Scholar
• Mišić VV (2020) Optimization of tree ensembles. Oper. Res. 68(5):1605–1624.Link, Google Scholar
• Murthy S, Kasif S, Salzberg S (1994) A system for induction of oblique decision trees. J Artificial Intelligence Res. 2(1):1–32.Crossref, Google Scholar
• Narodytska N, Ignatiev A, Pereira F, Marques-Silva J (2018) Learning optimal decision trees with SAT. Lang J, ed. Proc. Twenty-Seventh Internat. Joint Conf. Artificial Intelligence (IJCAI.org), 1362–1368.Google Scholar
• Ostrowski J, Linderoth J, Rossi F, Smriglio S (2011) Orbital branching. Math. Programming 126(1):147–178.Crossref, Google Scholar
• Quinlan J (1996) Machine Learning and ID3 (Morgan Kauffman, Los Altos, CA).Google Scholar
• Quinlan R (1993) C4.5: Programs for Machine Learning (Morgan Kaufmann Publishers, San Mateo, CA).Google Scholar
• Rudin C, Chen C, Chen Z, Huang H, Semenova L, Zhong C (2022) Interpretable machine learning: Fundamental principles and 10 grand challenges. Statist. Survey 16:1–85.Crossref, Google Scholar
• Sherali HD, Hobeika AG, Jeenanunta C (2009) An optimal constrained pruning strategy for decision trees. INFORMS J. Comput. 21(1):49–61.Link, Google Scholar
• Street WN (2005) Oblique multicategory decision trees using nonlinear programming. INFORMS J. Comput. 17(1):25–31.Link, Google Scholar
• Verhaeghe H, Nijssen S, Pesant G (2020) Learning optimal decision trees using constraint programming. Constraints 25(3):226–250.Crossref, Google Scholar
• Verwer S, Zhang Y (2019) Learning optimal classification trees using a binary linear program formulation. Proc. 33rd AAAI Conf. Artificial Intelligence (AAAI Press, Palo Alto, CA), 1625–1632.Google Scholar
• Yu J, Ignatiev A, Stuckey P, Le Bodic P (2020) Computing optimal decision sets with SAT. Simonis H, ed. Principles Practice Constraint Programming: 26th Internat. Conf., CP 2020, Lecture Notes in Computer Science, vol. 12333 (Springer, Cham, Switzerland), 952–970.Google Scholar
• Zhu H, Murali P, Phan D, Nguyen L, Kalagnanam J (2020) A scalable MIP-based method for learning optimal multivariate decision trees. Adv. Neural Inform. Processing Systems 33:1771–1781.Google Scholar