Using Common Random Numbers for Indifference-Zone Selection and Multiple Comparisons in Simulation

We present a general recipe for constructing experiment design and analysis procedures that simultaneously provide indifference-zone selection and multiple-comparison inference for choosing the best among k simulated systems. We then exhibit two such procedures that exploit the variance-reduction technique of common random numbers to reduce the sample size required to attain a fixed precision. One procedure is based on the Bonferroni inequality and is guaranteed to be statistically conservative. The other procedure is exact under a specific dependence structure, but may be slightly liberal otherwise. Both are easy to apply, requiring only simple calculations and tabled constants. We illustrate the procedures with a numerical example.