Fully Sequential Procedures for Large-Scale Ranking-and-Selection Problems in Parallel Computing Environments

Jun Luo
Jun Luo
[email protected]
Antai College of Economics and Management, Shanghai Jiao Tong University, Shanghai, China 200052
Search for more papers by this author
,
L. Jeff Hong
L. Jeff Hong
[email protected]
Department of Economics and Finance and Department of Management Sciences, College of Business, City University of Hong Kong, Kowloon, Hong Kong
Search for more papers by this author
,
Barry L. Nelson
Barry L. Nelson
[email protected]
Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, Illinois 60208
Search for more papers by this author
,
Yang Wu
Yang Wu
[email protected]
Tmall Company, the Alibaba Group, Hangzhou, Zhejiang, China 310000
Search for more papers by this author

Antai College of Economics and Management, Shanghai Jiao Tong University, Shanghai, China 200052

Department of Economics and Finance and Department of Management Sciences, College of Business, City University of Hong Kong, Kowloon, Hong Kong

Search for more papers by this author

Barry L. Nelson

[email protected]

Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, Illinois 60208

Search for more papers by this author

Yang Wu

[email protected]

Tmall Company, the Alibaba Group, Hangzhou, Zhejiang, China 310000

Search for more papers by this author

Published Online:18 Sep 2015https://doi.org/10.1287/opre.2015.1413

Supplemental Material

opre.2015.1413-sm.pdf (408 KB)

Cited by
- Fifty years of stochastic simulation: Where we are and where we need to go
  European Journal of Operational Research, Vol. 330, No. 3
- Ranking and selection in large-scale inference of heteroscedastic units
  The Annals of Applied Statistics, Vol. 20, No. 1
- Efficient Fully Sequential Indifference-Zone Procedures Using Properties of Multi-dimensional Brownian Motion Exiting a Sphere
  16 April 2025 | IISE Transactions, Vol. 58, No. 2
- A hybrid ranking and selection procedure to solve simulation-based discrete network design problems
  Transportation Research Part C: Emerging Technologies, Vol. 183
- PyPRS: A Python Software Package for Parallel Ranking and Selection Procedures
  Song Huang,
  Guangxin Jiang,
  Ying Zhong
  28 January 2026 | INFORMS Journal on Computing, Vol. 0, No. 0
- Enhanced Upper Confidence Bound Procedure for Large-Scale Ranking and Selection
- Simulation Optimization and Artificial Intelligence
  4 December 2025 | Foundations and Trends® in Optimization, Vol. 9, No. 4
- Adaptive sequential selection procedures for optimal quantile with control variates
  European Journal of Operational Research, Vol. 326, No. 3
- Blackbox Simulation Optimization
  30 July 2024 | Journal of the Operations Research Society of China, Vol. 13, No. 3
- Simulation Budget Allocation for Improving Scheduling and Routing of Automated Guided Vehicles in Warehouse Management
  29 July 2024 | Journal of the Operations Research Society of China, Vol. 13, No. 3
- Review of Large-Scale Simulation Optimization
  18 June 2025 | Journal of the Operations Research Society of China, Vol. 13, No. 3
- A budget-adaptive allocation rule for optimal computing budget allocation
  European Journal of Operational Research, Vol. 325, No. 2
- Stochastic Localization Methods for Convex Discrete Optimization via Simulation
  Haixiang Zhang; ,
  Zeyu Zheng,
  Javad Lavaei
  6 October 2023 | Operations Research, Vol. 73, No. 2
- The (Surprising) Sample Optimality of Greedy Procedures for Large-Scale Ranking and Selection
  Zaile Li; ,
  Weiwei Fan; ,
  L. Jeff Hong
  7 May 2024 | Management Science, Vol. 71, No. 2
- Reference alternatives based knockout-tournament procedure for ranking and selection
  European Journal of Operational Research, Vol. 320, No. 3
- Blackbox Simulation Optimization
  1 January 2025 | SSRN Electronic Journal, Vol. 72
- Simulation Optimization: An Introductory Tutorial on Methodology
- Fixed-Precision Ranking and Selection as Markov Decision Process
- A Preliminary Study on Accelerating Simulation Optimization With GPU Implementation
- Enhancing Language Model with Both Human and Artificial Intelligence Feedback Data
- Marginal improvement procedures for top- m selection
  Automatica, Vol. 170
- Rebooting simulation
  8 November 2023 | IISE Transactions, Vol. 56, No. 4
- Efficient Learning for Clustering and Optimizing Context-Dependent Designs
  Haidong Li,
  Henry Lam,
  Yijie Peng
  23 September 2022 | Operations Research, Vol. 72, No. 2
- Data-Driven Ranking and Selection Under Input Uncertainty
  Di Wu,
  Yuhao Wang,
  Enlu Zhou
  18 October 2022 | Operations Research, Vol. 72, No. 2
- Real-time digital twin-based optimization with predictive simulation learning
  7 March 2022 | Journal of Simulation, Vol. 18, No. 1
- Parallel Adaptive Survivor Selection
  Linda Pei,
  Barry L. Nelson,
  Susan R. Hunter
  24 August 2022 | Operations Research, Vol. 72, No. 1
- POMDP-Based Ranking and Selection
- Screening Simulated Systems for Optimization
- Gradient-Based Algorithms for Convex Discrete Optimization via Simulation
  Haixiang Zhang,
  Zeyu Zheng,
  Javad Lavaei
  28 April 2022 | Operations Research, Vol. 71, No. 5
- Adaptive fully sequential selection procedures with linear and nonlinear control variates
  16 June 2022 | IISE Transactions, Vol. 55, No. 6
- Efficient estimation of a risk measure requiring two-stage simulation optimization
  European Journal of Operational Research, Vol. 305, No. 3
- Optimal budget allocation policy for tabu search in stochastic simulation optimization
  Computers & Operations Research, Vol. 150
- Simulation Optimization: Discrete Optimization via Simulation
  3 May 2023
- Simulation Optimization: Multi-fidelity Optimization
  30 March 2023
- Let's Do Ranking & Selection
- Solving Large-Scale Fixed-Budget Ranking and Selection Problems
  L. Jeff Hong,
  Guangxin Jiang,
  Ying Zhong
  28 July 2022 | INFORMS Journal on Computing, Vol. 34, No. 6
- Posterior-Based Stopping Rules for Bayesian Ranking-and-Selection Procedures
  David J. Eckman,
  Shane G. Henderson
  19 January 2022 | INFORMS Journal on Computing, Vol. 34, No. 3
- Efficient learning for decomposing and optimizing random networks
  Fundamental Research, Vol. 2, No. 3
- Canonical Problems and Applications
  8 April 2022
- Preventive maintenance for heterogeneous parallel systems with two failure modes
  Reliability Engineering & System Safety, Vol. 220
- Speeding Up Paulson’s Procedure for Large-Scale Problems Using Parallel Computing
  Ying Zhong,
  Shaoxuan Liu,
  Jun Luo,
  L. Jeff Hong
  4 June 2021 | INFORMS Journal on Computing, Vol. 34, No. 1
- Dynamic Sampling Allocation Under Finite Simulation Budget for Feasibility Determination
  Zhongshun Shi,
  Yijie Peng,
  Leyuan Shi,
  Chun-Hung Chen,
  Michael C. Fu
  23 March 2021 | INFORMS Journal on Computing, Vol. 34, No. 1
- Knockout-Tournament Procedures for Large-Scale Ranking and Selection in Parallel Computing Environments
  Ying Zhong,
  L. Jeff Hong
  4 June 2021 | Operations Research, Vol. 70, No. 1
- Foundations of Ranking & Selection for Simulation Optimization
  1 December 2022
- Surrogate-Based Simulation Optimization
  L. Jeff Hong,
  Xiaowei Zhang
  18 October 2021
- Ranking and Selection with Covariates for Personalized Decision Making
  Haihui Shen,
  L. Jeff Hong,
  Xiaowei Zhang
  12 February 2021 | INFORMS Journal on Computing, Vol. 33, No. 4
- Emerging approaches to support dynamic mission planning: survey and recommendations for future research
  7 February 2020 | The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology, Vol. 18, No. 4
- Review on ranking and selection: A new perspective
  29 March 2021 | Frontiers of Engineering Management, Vol. 8, No. 3
- Mixed-input Gaussian process emulators for computer experiments with a large number of categorical levels
  24 June 2020 | Journal of Quality Technology, Vol. 53, No. 4
- Ranking and selection for terminating simulation under sequential sampling
  3 August 2020 | IISE Transactions, Vol. 53, No. 7
- A Study on Efficient Computing Budget Allocation for a Two-Stage Problem
  5 January 2021 | Asia-Pacific Journal of Operational Research, Vol. 38, No. 02
- Three Views of Simulation
  27 August 2021
- Asymptotic efficiency analysis on the modified Paulson’s procedure with a PAC guarantee
  Operations Research Letters, Vol. 49, No. 1
- Context-Dependent Ranking and Selection under a Bayesian Framework
- Parallel computational optimization in operations research: A new integrative framework, literature review and research directions
  European Journal of Operational Research, Vol. 287, No. 1
- Information theory for ranking and selection
  14 April 2020 | Naval Research Logistics (NRL), Vol. 67, No. 4
- Stochastic Control Framework for Determining Feasible Alternatives in Sampling Allocation
  IEEE Transactions on Automatic Control, Vol. 65, No. 6
- Algorithm for Calculating the Initial Sample Size in a Fully Sequential Ranking and Selection Procedure
  6 May 2020 | Asia-Pacific Journal of Operational Research, Vol. 37, No. 03
- Selecting the Best Simulated System: Thinking Differently About an Old Problem
  2 May 2020
- Efficient Simulation Budget Allocation With Bound Information
  IEEE Transactions on Automatic Control, Vol. 65, No. 1
- Discrete Convex Simulation Optimization
  SSRN Electronic Journal, Vol. 27
- Stochastic Localization Methods for Discrete Convex Simulation Optimization
  SSRN Electronic Journal, Vol. 51
- Heterogeneous Assets Market Design
- Dynamic Sampling Procedure for Decomposable Random Networks
- A Spectral Index for Selecting the Best Alternative
- Offline Simulation Online Application: A New Framework of Simulation-Based Decision Making
  9 December 2019 | Asia-Pacific Journal of Operational Research, Vol. 36, No. 06
- Bayesian sequential data collection for stochastic simulation calibration
  European Journal of Operational Research, Vol. 277, No. 1
- optimizing outpatient Department Staffing Level using Multi-Fidelity Models
- Efficient Simulation Sampling Allocation Using Multifidelity Models
  IEEE Transactions on Automatic Control, Vol. 64, No. 8
- A unified framework for stochastic optimization
  European Journal of Operational Research, Vol. 275, No. 3
- Customer Incentive Rebalancing Plan in Free-Float Bike-Sharing System with Limited Information
  31 May 2019 | Sustainability, Vol. 11, No. 11
- An efficient simulation procedure for ranking the top simulated designs in the presence of stochastic constraints
  Automatica, Vol. 103
- Distributed simulation: state-of-the-art and potential for operational research
  European Journal of Operational Research, Vol. 273, No. 1
- A REVIEW OF STATIC AND DYNAMIC OPTIMIZATION FOR RANKING AND SELECTION
- Simulation Budget Allocation for Selecting the Top-m Designs With Input Uncertainty
  IEEE Transactions on Automatic Control, Vol. 63, No. 9
- Dynamic Sampling for Feasibility Determination
- On parallel policies for ranking and selection problems
  17 October 2017 | Journal of Applied Statistics, Vol. 45, No. 9
- Sensitivity and covariance in stochastic complementarity problems with an application to North American natural gas markets
  European Journal of Operational Research, Vol. 268, No. 1
- History of seeking better solutions, AKA simulation optimization
- Optimal computing budget allocation via sampling based unbiased approximation
- Optimal design of master-worker architecture for parallelized simulation optimization
- Simulation budget allocation for simultaneously selecting the best and worst subsets
  Automatica, Vol. 84
- Combined variance reduction techniques in fully sequential selection procedures
  13 December 2017 | Naval Research Logistics (NRL), Vol. 64, No. 6
- Efficient Ranking and Selection in Parallel Computing Environments
  Eric C. Ni,
  Dragos F. Ciocan,
  Shane G. Henderson,
  Susan R. Hunter
  7 March 2017 | Operations Research, Vol. 65, No. 3
- Parallel Ranking and Selection
  29 August 2017
- Parallel Empirical Stochastic Branch and Bound for large-scale discrete optimization via Simulation
- Speeding up pairwise comparisons for large scale ranking and selection
- Simulation optimization in the era of Industrial 4.0 and the Industrial Internet
  19 December 2017 | Journal of Simulation, Vol. 10, No. 4
- Integrated Modeling for Location Analysis
  18 October 2016 | Foundations and Trends® in Technology, Information and Operations Management, Vol. 9, No. 1-2
- ‘Some tactical problems in digital simulation’ for the next 10 years
  19 December 2017 | Journal of Simulation, Vol. 10, No. 1

Volume 63, Issue 5

September-October 2015

Pages ii-iv, 979-1244

Article Information

Supplemental Material

Metrics

Information

Received:June 01, 2013
Accepted:June 01, 2015
Published Online:September 18, 2015

Cite as

Jun Luo, L. Jeff Hong, Barry L. Nelson, Yang Wu (2015) Fully Sequential Procedures for Large-Scale Ranking-and-Selection Problems in Parallel Computing Environments. Operations Research 63(5):1177-1194.

https://doi.org/10.1287/opre.2015.1413

Keywords

PDF download

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Fully Sequential Procedures for Large-Scale Ranking-and-Selection Problems in Parallel Computing Environments

Supplemental Material

Volume 63, Issue 5

Article Information

Supplemental Material

Metrics

Information

Cite as

Keywords

Sign Up for INFORMS Publications Updates and News