Open Access

Fast Best Subset Selection: Coordinate Descent and Local Combinatorial Optimization Algorithms

Hussein Hazimeh
Corresponding Author
Hussein Hazimeh
[email protected]
https://orcid.org/0000-0003-4501-0678
Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139;
Search for more papers by this author
,
Rahul Mazumder
Rahul Mazumder
[email protected]
https://orcid.org/0000-0003-1384-9743
Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139;MIT Sloan School of Management, Massachusetts Institute of Technology, Cambridge, Massachusetts 02142;
Search for more papers by this author

Hussein Hazimeh

Corresponding Author

Hussein Hazimeh

[email protected]

https://orcid.org/0000-0003-4501-0678

Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139;

Search for more papers by this author

Rahul Mazumder

[email protected]

https://orcid.org/0000-0003-1384-9743

Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139;MIT Sloan School of Management, Massachusetts Institute of Technology, Cambridge, Massachusetts 02142;

Search for more papers by this author

Published Online:13 Aug 2020https://doi.org/10.1287/opre.2019.1919

Supplemental Material

opre.2019.1919.sm1.pdf

Cited by
- All-integer global optimization for high-dimensional sparse regression, with applications in financial and genomic data
  European Journal of Operational Research, Vol. 334, No. 2
- Adaptive reweighting for joint estimation of sparse and non-sparse components
  Journal of Statistical Planning and Inference, Vol. 81
- Sparse Gaussian Graphical Models with Discrete Optimization: Computational and Statistical Perspectives
  Kayhan Behdin,
  Wenyu Chen,
  Rahul Mazumder
  9 April 2026 | Operations Research, Vol. 0, No. 0
- Modelling with categorical features via exact fusion and sparsity regularization
  9 April 2026 | Journal of the Royal Statistical Society Series B: Statistical Methodology, Vol. 44
- Personalised incentives for demand management of congested public transport systems: A reverse-engineering approach and application
  Transportation Research Part C: Emerging Technologies, Vol. 185
- Randomized smart subset selection for high-dimensional generalized linear models with false positive control
  25 March 2026 | Advances in Data Analysis and Classification, Vol. 44
- DIRECTION IDENTIFICATION AND MINIMAX ESTIMATION IN HIGH-DIMENSIONAL SPARSE REGRESSION VIA A GENERALIZED EIGENVALUE APPROACH
  24 February 2026 | Econometric Theory, Vol. 74
- Best-subset instrumental variable selection method using mixed integer optimization with applications to health-related quality of life and education–wage analyses
  1 November 2025 | Statistics and Computing, Vol. 36, No. 1
- Integer programming in psychology: A review and directions for future research
  26 February 2025 | British Journal of Mathematical and Statistical Psychology, Vol. 79, No. 1
- Joint estimation of sparse and dense components through structured iteration
  28 December 2025 | TEST, Vol. 81
- Multi-task learning for sparsity pattern heterogeneity: statistical and computational perspectives
  17 December 2025 | Journal of the Royal Statistical Society Series B: Statistical Methodology, Vol. 22
- Outlier Detection in Regression: Conic Quadratic Formulations
  Andrés Gómez,
  José Neto
  2 December 2025 | INFORMS Journal on Computing, Vol. 0, No. 0
- Automated subset selection via information criteria optimization in generalized linear models
  15 October 2025 | Computational Statistics, Vol. 40, No. 9
- Convex mixed-integer optimization with Frank–Wolfe methods
  28 June 2025 | Mathematical Programming Computation, Vol. 17, No. 4
- A cell type and state specific gene regulation network inference method for immune regulatory analysis
  13 August 2025 | npj Systems Biology and Applications, Vol. 11, No. 1
- Integrated truss optimization using a ternary‐quantized gradient method with implicit topology control
  Computer-Aided Civil and Infrastructure Engineering, Vol. 40, No. 31
- An unsupervised tool for biomarker discovery and cancer subtyping applied to glioblastoma
  22 December 2025 | BioData Mining, Vol. 18, No. 1
- Rethinking Hard Thresholding Pursuit: Full Adaptation and Sharp Estimation
  IEEE Transactions on Information Theory, Vol. 71, No. 11
- Responsible Machine Learning via Mixed-Integer Optimization
  Nathan Justin,
  Qingshi Sun,
  Andrés Gómez,
  Phebe Vayanos
  6 October 2025
- Feature Selection and Grouping Effect Analysis for Credit Evaluation via Regularized Diagonal Distance Metric Learning
  Tie Li,
  Gang Kou,
  Yi Peng,
  Philip S. Yu
  11 December 2024 | INFORMS Journal on Computing, Vol. 37, No. 5
- An attention-based fuzzy CNN-LTSM network for visual object recognition from fMRI images
  5 August 2025 | Journal of Computational Neuroscience, Vol. 53, No. 3
- Optimizing multivariate adaptive regression splines (MARS) with coordinate descent to accurately select the best model for house price prediction
  25 August 2025 | Discover Applied Sciences, Vol. 7, No. 9
- Integrated subset selection and bandwidth estimation algorithm for geographically weighted regression
  Pattern Recognition, Vol. 165
- Proximal Distance Algorithms for Sparse Portfolio Selections
  10 June 2025 | Journal of the Operations Research Society of China, Vol. 37
- Optimal forecast reconciliation with time series selection
  European Journal of Operational Research, Vol. 323, No. 2
- Robust estimation of regression models with potentially endogenous outliers via a modern optimization lens
  3 April 2025 | Econometric Reviews, Vol. 27
- Feature and functional form selection in additive models via mixed-integer optimization
  Computers & Operations Research, Vol. 176
- Numerical evaluation of the good subset approach to variable selection
  24 March 2025 | Communications in Statistics - Simulation and Computation, Vol. 20
- Multi-model subset selection
  Computational Statistics & Data Analysis, Vol. 203
- Predicting census survey response rates with parsimonious additive models and structured interactions
  The Annals of Applied Statistics, Vol. 19, No. 1
- Coordinate-update algorithms can efficiently detect infeasible optimization problems
  Journal of Mathematical Analysis and Applications, Vol. 543, No. 2
- Best Subset Solution Path for Linear Dimension Reduction Models Using Continuous Optimization
  16 December 2024 | Biometrical Journal, Vol. 67, No. 1
- An augmented Lagrangian approach for cardinality constrained minimization applied to variable selection problems
  Applied Numerical Mathematics, Vol. 208
- PolyCD: Optimization via Cycling through the Vertices of a Polytope
  17 October 2024 | SIAM Journal on Optimization, Vol. 34, No. 4
- LIVE-ITS: LSH-based Interactive Visualization Explorer for Large-Scale Incomplete Time Series
- Continuous Optimization for Offline Change Point Detection and Estimation
- On Sure Early Selection of the Best Subset
  IEEE Transactions on Information Theory, Vol. 70, No. 12
- Water source, latrine type, and rainfall are associated with detection of non-optimal and enteric bacteria in the vaginal microbiome: a prospective observational cohort study nested within a cluster randomized controlled trial
  18 December 2024 | BMC Infectious Diseases, Vol. 24, No. 1
- Optimal variable identification for accurate detection of causal expression Quantitative Trait Loci with applications in heart-related diseases
  5 June 2024 | Computational and Structural Biotechnology Journal, Vol. 23
- Constrained optimization of rank-one functions with indicator variables
  20 January 2024 | Mathematical Programming, Vol. 208, No. 1-2
- Group Selection and Shrinkage: Structured Sparsity for Semiparametric Additive Models
  22 April 2024 | Journal of Computational and Graphical Statistics, Vol. 33, No. 4
- Integrative prediction model for radiation pneumonitis incorporating genetic and clinical-pathological factors using machine learning
  Clinical and Translational Radiation Oncology, Vol. 48
- FAST: An Optimization Framework for Fast Additive Segmentation in Transparent ML
  24 August 2024
- Fast and scalable ensemble learning method for versatile polygenic risk prediction
  7 August 2024 | Proceedings of the National Academy of Sciences, Vol. 121, No. 33
- Exterior-Point Optimization for Sparse and Low-Rank Optimization
  26 May 2024 | Journal of Optimization Theory and Applications, Vol. 202, No. 2
- Mixed-Integer Linear Optimization Formulations for Feature Subset Selection in Kernel SVM Classification
  IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, Vol. E107.A, No. 8
- Subset Selection in Support Vector Machine
- SRMD: Sparse Random Mode Decomposition
  20 June 2023 | Communications on Applied Mathematics and Computation, Vol. 6, No. 2
- Estimation of a treatment effect based on a modified covariates method with $$L_0$$ norm
  9 February 2024 | Artificial Life and Robotics, Vol. 29, No. 2
- Signal-to-Noise Ratio Aware Minimaxity and Higher-Order Asymptotics
  IEEE Transactions on Information Theory, Vol. 70, No. 5
- Cardinality Minimization, Constraints, and Regularization: A Survey
  8 August 2024 | SIAM Review, Vol. 66, No. 3
- Physiochemical machine learning models predict operational lifetimes of CH 3 NH 3 PbI 3 perovskite solar cells
  1 January 2024 | Journal of Materials Chemistry A, Vol. 12, No. 16
- COMBSS: best subset selection via continuous optimization
  12 February 2024 | Statistics and Computing, Vol. 34, No. 2
- Subset Selection and the Cone of Factor-Width- k Matrices
  22 February 2024 | SIAM Journal on Optimization, Vol. 34, No. 1
- Best Principal Submatrix Selection for the Maximum Entropy Sampling Problem: Scalable Algorithms and Performance Guarantees
  Yongchun Li,
  Weijun Xie
  30 May 2023 | Operations Research, Vol. 72, No. 2
- Variable selection in additive models via hierarchical sparse penalty
  3 February 2023 | Canadian Journal of Statistics, Vol. 52, No. 1
- Comparing solution paths of sparse quadratic minimization with a Stieltjes matrix
  4 May 2023 | Mathematical Programming, Vol. 204, No. 1-2
- Generating collective counterfactual explanations in score-based classification via mathematical optimization
  Expert Systems with Applications, Vol. 238
- Efficiency of Identification of Blackcurrant Powders Using Classifier Ensembles
  24 February 2024 | Foods, Vol. 13, No. 5
- A neutral comparison of algorithms to minimize L0 penalties for high‐dimensional variable selection
  8 July 2023 | Biometrical Journal, Vol. 66, No. 1
- Best subset selection with shrinkage: sparse additive hazards regression with the grouping effect
  4 July 2023 | Journal of Statistical Computation and Simulation, Vol. 93, No. 18
- HARFE: hard-ridge random feature expansion
  9 August 2023 | Sampling Theory, Signal Processing, and Data Analysis, Vol. 21, No. 2
- Integration of single sample and population analysis for understanding immune evasion mechanisms of lung cancer
  10 February 2023 | npj Systems Biology and Applications, Vol. 9, No. 1
- Innovative approaches to addressing the tradeoff between interpretability and accuracy in ship fuel consumption prediction
  Transportation Research Part C: Emerging Technologies, Vol. 157
- Dynamic Covariance Estimation under Structural Assumptions via a Joint Optimization Approach
  25 November 2023
- A swap-based sparse array design method in personal sound zone
  2 November 2023 | The Journal of the Acoustical Society of America, Vol. 154, No. 5
- inferCSN: Inferring Cell-Specific Gene Regulatory Network
  CRAN: Contributed Packages
- ${\ell_0}$ Optimization with Robust Non-Oracular Quantum Search
  19 October 2023 | Technologies, Vol. 11, No. 5
- A comparison of logistic regression methods for Ising model estimation
  20 October 2022 | Behavior Research Methods, Vol. 55, No. 7
- Safe Peeling for $\ell_{0}$-Regularized Least-Squares
- Carousel Greedy Algorithms for Feature Selection in Linear Regression
  19 September 2023 | Algorithms, Vol. 16, No. 9
- Sparse quantile regression
  Journal of Econometrics, Vol. 235, No. 2
- On clustering and interpreting with rules by means of mathematical optimization
  Computers & Operations Research, Vol. 154
- Fiducial variable selection for the high-dimensional regression model
  31 March 2023 | SCIENTIA SINICA Mathematica, Vol. 53, No. 6
- OSCAR: Optimal subset cardinality regression using the L0-pseudonorm with applications to prognostic modelling of prostate cancer
  10 March 2023 | PLOS Computational Biology, Vol. 19, No. 3
- Linear regression with partially mismatched data: local search with theoretical guarantees
  17 August 2022 | Mathematical Programming, Vol. 197, No. 2
- Grouped variable selection with discrete optimization: Computational and statistical perspectives
  The Annals of Statistics, Vol. 51, No. 1
- Efficient Gradient Support Pursuit With Less Hard Thresholding for Cardinality-Constrained Learning
  IEEE Transactions on Neural Networks and Learning Systems, Vol. 33, No. 12
- An Alternating Method for Cardinality-Constrained Optimization: A Computational Study for the Best Subset Selection and Sparse Portfolio Problems
  Carina Moreira Costa,
  Dennis Kreber,
  Martin Schmidt
  8 August 2022 | INFORMS Journal on Computing, Vol. 34, No. 6
- Sparse regression at scale: branch-and-bound rooted in first-order optimization
  18 October 2021 | Mathematical Programming, Vol. 196, No. 1-2
- Subset selection in network-linked data
  1 February 2022 | Journal of Statistical Computation and Simulation, Vol. 92, No. 11
- The backbone method for ultra-high dimensional sparse machine learning
  22 January 2022 | Machine Learning, Vol. 111, No. 6
- Robust subset selection
  Computational Statistics & Data Analysis, Vol. 169
- Tipping the Balance: Vitamin D Inadequacy in Children Impacts the Major Gut Bacterial Phyla
  26 January 2022 | Biomedicines, Vol. 10, No. 2
- ProSGPV: an R package for variable selection with second-generation p-values
  18 January 2022 | F1000Research, Vol. 11
- Sparse classification: a scalable discrete optimization perspective
  2 November 2021 | Machine Learning, Vol. 110, No. 11-12
- An extended Newton-type algorithm for ℓ2 -regularized sparse logistic regression and its efficiency for classifying large-scale datasets
  Journal of Computational and Applied Mathematics, Vol. 397
- An $L_0$ Regularization Method for Imaging Genetics and Whole Genome Association Analysis on Alzheimer's Disease
  IEEE Journal of Biomedical and Health Informatics, Vol. 25, No. 9
- Lasry-Lions Envelopes and Nonconvex Optimization: A Homotopy Approach
- The Trimmed Lasso: Sparse Recovery Guarantees and Practical Optimization by the Generalized Soft-Min Penalty
  9 September 2021 | SIAM Journal on Mathematics of Data Science, Vol. 3, No. 3
- Sparse Regression in Cancer Genomics: Comparing Variable Selection and Predictions in Real World Data
  27 November 2021 | Cancer Informatics, Vol. 20
- Novel mixed integer optimization sparse regression approach in chemometrics
  Analytica Chimica Acta, Vol. 1137
- Rejoinder: Best Subset, Forward Stepwise or Lasso? Analysis and Recommendations Based on Extensive Comparisons
  Statistical Science, Vol. 35, No. 4
- Discussion of “Best Subset, Forward Stepwise or Lasso? Analysis and Recommendations Based on Extensive Comparisons”
  Statistical Science, Vol. 35, No. 4
- Matrix completion with nonconvex regularization: spectral operators and scalable algorithms
  14 March 2020 | Statistics and Computing, Vol. 30, No. 4
- Randomized Gradient Boosting Machine
  7 October 2020 | SIAM Journal on Optimization, Vol. 30, No. 4
- L0Learn: Fast Algorithms for Best Subset Selection
  CRAN: Contributed Packages

Volume 68, Issue 5

September-October 2020

Pages iii-vi, 1285-1624, C2-C3

Article Information

Supplemental Material

Metrics

Information

Received:April 10, 2018
Accepted:June 30, 2019
Published Online:August 13, 2020

Cite as

Hussein Hazimeh, Rahul Mazumder (2020) Fast Best Subset Selection: Coordinate Descent and Local Combinatorial Optimization Algorithms. Operations Research 68(5):1517-1537.

https://doi.org/10.1287/opre.2019.1919

Keywords

PDF download

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Fast Best Subset Selection: Coordinate Descent and Local Combinatorial Optimization Algorithms

Supplemental Material

Volume 68, Issue 5

Article Information

Supplemental Material

Metrics

Information

Cite as

Keywords

Sign Up for INFORMS Publications Updates and News