An Efficient Morris Method-Based Framework for Simulation Factor Screening

References

• Ankenman BE, Cheng RCH, Lewis SM (2014) Screening for dispersion effects by sequential bifurcation. ACM Trans. Model. Comput. Simulation 25(1):1–27.Crossref, Google Scholar
• Barton RR, Nelson BL, Xie W (2014) Quantifying input uncertainty via simulation confidence intervals. INFORMS J. Comput. 26(1):74–87.Link, Google Scholar
• Bass AJ, Dabney A, Robinson D (2015) Qvalue: Q-value estimation for false discovery rate control. R package version 2.4.2. Accessed January 1, 2016, http://github.com/jdstorey/qvalue.Google Scholar
• Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B: Methodology 57(1):289–300.Google Scholar
• Bettonvil B, Kleijnen JPC (1996) Searching for important factors in simulation models with many factors: Sequential bifurcation. Eur. J. Oper. Res. 96(1):180–194.Crossref, Google Scholar
• Borgonovo E, Plischke E (2016) Sensitivity analysis: A review of recent advances. Eur. J. Oper. Res. 248(3):869–887.Crossref, Google Scholar
• Boukouvalas A, Gosling JP, Maruri-Aguilar H (2014) An efficient screening method for computer experiments. Technometrics 56(4):422–431.Crossref, Google Scholar
• Broadie M, Du Y, Moallemi CC (2011) Efficient risk estimation via nested sequential simulation. Management Sci. 57(6):1172–1194.Link, Google Scholar
• Campolongo F, Braddock R (1999) The use of graph theory in the sensitivity analysis of the model output: A second order screening method. Reliability Engrg. System Safety 64(1):1–12.Crossref, Google Scholar
• Campolongo F, Cariboni J, Saltelli A (2007) An effective screening design for sensitivity analysis of large models. Environ. Model. Software 22(10):1509–1518.Crossref, Google Scholar
• Carpenter JR, Goldstein H, Rasbash J (2003) A novel bootstrap procedure for assessing the relationship between class size and achievement. J. Roy. Statist. Soc. Ser. C: Appl. Statist. 52(4):431–443.Crossref, Google Scholar
• Chambers R, Chandra H (2013) A random effect block bootstrap for clustered data. J. Comput. Graphical Statist. 22(2):452–470.Crossref, Google Scholar
• Chang PB, Williams BJ, Santner TJ, Notz WI, Bartel DL (1999) Robust optimization of total joint replacements incorporating environmental variables. J. Biomechanical Engrg. 121(3):304–310.Crossref, Google Scholar
• Cheng RCH (1997) Searching for important factors: Sequential bifurcation under uncertainty. Andradóttir S, Healy KJ, Withers DH, Nelson BL, eds. Proc. 1997 Winter Simulation Conf. (IEEE, Piscataway, NJ), 275–280.Crossref, Google Scholar
• Cochran WG (1977) Sampling Techniques, 3rd ed. (John Wiley & Sons, New York).Google Scholar
• Cropp RA, Braddock RD (2002) The new Morris method: An efficient second-order screening method. Reliability Engrg. System Safety 78(1):77–83.Crossref, Google Scholar
• Davidson R, MacKinnon JG (2000) Bootstrap tests: How many bootstraps? Econometric Rev. 19(1):55–68.Crossref, Google Scholar
• Draguljić D, Woods DC, Dean AM, Lewis SM, Vine AE (2014) Screening strategies in the presence of interactions (with discussion). Technometrics 56(1):1–28.Crossref, Google Scholar
• Efron B (2004) Large-scale simultaneous hypothesis testing. J. Amer. Statist. Assoc. 99(465):96–104.Crossref, Google Scholar
• Efron B (2007) Correlation and large-scale simultaneous significance testing. J. Amer. Statist. Assoc. 102(477):93–103.Crossref, Google Scholar
• Efron B, Tibshirani RJ (1993) An Introduction to the Bootstrap, vol. 57 (Chapman Hall, New York).Crossref, Google Scholar
• Efron B, Tibshirani R, Storey JD, Tusher V (2001) Empirical Bayes analysis of a microarray experiment. J. Amer. Statist. Assoc. 96(456):1151–1160.Crossref, Google Scholar
• Fédou J, Rendas MJ (2015) Extending Morris method: Identification of the interaction graph using cycle-equitable designs. J. Statist. Comput. Simulation 85(7):1398–1419.Crossref, Google Scholar
• Field CA, Welsh AH (2007) Bootstrapping clustered data. J. Roy. Statist. Soc. Ser. B: Statist. Methodology 69(3):369–390.Crossref, Google Scholar
• Frazier PI, Jedynak B, Chen L (2012) Sequential screening: A Bayesian dynamic programming analysis of optimal group-splitting. Laroque C, Himmelspach J, Pasupathy R, Rose O, Uhrmacher AM, eds. Proc. 2012 Winter Simulation Conf. (IEEE, Piscataway, NJ), 1–12.Crossref, Google Scholar
• Friedman LW, Friedman HH (1995) Analyzing simulation output using the bootstrap method. Simulation 64(2):95–100.Crossref, Google Scholar
• Han S, Andrei A, Tsui K (2015) Efficient stratified testing procedure for a false discovery rate. Comm. Statist. Simulation Comput. 44(5):1117–1125.Crossref, Google Scholar
• Hargreaves JC, Annan JD, Edwards NR, Marsh R (2004) Efficient climate forecasting method using an intermediate complexity earth system model and the ensemble Kalman filter. Climate Dynam. 23(7–8):745–760.Crossref, Google Scholar
• Hu JX, Zhao H, Zhou HH (2010) False discovery rate control with groups. J. Amer. Statist. Assoc. 105(491):1215–1227.Crossref, Google Scholar
• Jin J, Cai TT (2007) Estimating the null and the proportion of nonnull effects in large-scale multiple comparisons. J. Amer. Statist. Assoc. 102(478):495–506.Crossref, Google Scholar
• Kennedy M, Anderson C, O’Hagan A, Lomas M, Woodward F, Gosling J, Heinemeyer A (2008) Quantifying uncertainty in the biospheric carbon flux for England and Wales. J. Roy. Statist. Soc. Ser. A: Statist. Soc. 171(1):109–135.Crossref, Google Scholar
• Kleijnen JPC (2015) Design and Analysis of Simulation Experiments, 2nd ed. (Springer, New York).Crossref, Google Scholar
• Kleijnen JPC, Bettonvil B, Persson F (2006) Screening for the important factors in large discrete-event simulation models: Sequential bifurcation and its applications. Dean A, Lewis S, eds. Screening: Methods for Experimentation in Industry, Drug Discovery, and Genetic (Springer, New York), 287–307.Crossref, Google Scholar
• Kleijnen JPC, Pierreval H, Zhang J (2011) Methodology for determining the acceptability of system designs in uncertain environments. Eur. J. Oper. Res. 209(2):176–183.Crossref, Google Scholar
• Kleijnen JPC, Sanchez SM, Lucas TW, Cioppa TM (2005) State-of-the-art review: A user’s guide to the brave new world of designing simulation experiments. INFORMS J. Comput. 17(3):263–289.Link, Google Scholar
• Linkletter C, Bingham D, Hengartner N, Ye KQ (2006) Variable selection for Gaussian process models in computer experiments. Technometrics 48(4):478–490.Crossref, Google Scholar
• Lucas TW, Sanchez SM, Martinez F, Sickinger LR, Roginski JW (2007) Defense and homeland security applications of multi-agent simulations. Henderson SG, Biller B, Hsieh MH, Shortle J, Tew JD, Barton RR, eds. Proc. 2007 Winter Simulation Conf. (IEEE Piscataway, NJ), 138–149.Crossref, Google Scholar
• MacKinnon JG (2009) Bootstrap hypothesis testing. Belsley DA, Kontoghiorghes EJ, eds. Handbook of Computational Econometrics (John Wiley & Sons, New York), 183–213.Crossref, Google Scholar
• Markoff J (2008) Military supercomputer sets record. New York Times (June 9), http://www.nytimes.com/2008/06/09/technology/09petaflops.html?_r=0.Google Scholar
• Meinshausen N, Rice J (2006) Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses. Ann. Statist. 34(1):373–393.Crossref, Google Scholar
• Montgomery DC (2017) Design and Analysis of Experiments, 8th ed. (John Wiley & Sons, New York).Google Scholar
• Moon H, Dean AM, Santner TJ (2012) Two-stage sensitivity-based group screening in computer experiments. Technometrics 54(4):376–387.Crossref, Google Scholar
• Morris MD (1991) Factorial sampling plans for preliminary computational experiments. Technometrics 33(2):161–174.Crossref, Google Scholar
• Morris MD (2006) Input screening: Finding the important model inputs on a budget. Reliability Engrg. Syst. Safety 91(10/11):1252–1256.Crossref, Google Scholar
• Morris MD (2010) Design of Experiments: An Introduction Based on Linear Models (Chapman & Hall/CRC, Boca Raton, FL).Crossref, Google Scholar
• Phoa FKH, Wong WK, Xu H (2009) The need of considering the interactions in the analysis of screening designs. J. Chemometrics 23(10):545–553.Crossref, Google Scholar
• Phoa FKH, Chen RB, Wang W, Wong WK (2016) Optimizing two-level supersaturated designs using swarm intelligence techniques. Technometrics 58(1):43–49.Crossref, Google Scholar
• Pruyt E, Cunningham S, Kwakkel JH, De Bruijn JA (2014) From data-poor to data-rich: System dynamics in the era of big data. Proc. 32nd Internat. Conf. System Dynam. Soc. (System Dynamics Society, Delft, Netherlands), 1–12.Google Scholar
• Pujol G (2009) Simplex-based screening designs for estimating metamodels. Reliability Engrg. Syst. Safety 94(7):1156–1160.Crossref, Google Scholar
• Ren S, Lai H, Tong W, Aminzadeh M, Hou X, Lai S (2010) Nonparametric bootstrapping for hierarchical data. J. Appl. Statist. 37(9):1487–1498.Crossref, Google Scholar
• Sanchez SM (2014) Simulation experiments: Better data, not just big data. Tolk A, Diallo SD, Ryzhov IO, Yilmaz L, Buckley S, Miller JA, eds. Proc. 2014 Winter Simulation Conf. (IEEE, Piscataway, NJ), 805–816.Crossref, Google Scholar
• Sanchez SM, Moeeni F, Sanchez PJ (2006) So many factors, so little time…simulation experiments in the frequency domain. Internat. J. Production Econom. 103(1):149–165.Crossref, Google Scholar
• Sanchez SM, Wan H, Lucas TW (2009) A two-phase screening procedure for simulation experiments. ACM Trans. Model. Comput. Simulation 19(2):1–24.Crossref, Google Scholar
• Sanchez SM, Lucas TW, Sanchez PJ, Nannini CJ, Wan H (2012) Designs for Large-Scale Simulation Experiments, with Applications to Defense and Homeland Security (John Wiley & Sons, Hoboken, NJ), 413–441.Crossref, Google Scholar
• Schruben LW, Cogliano VJ (1987) An experimental procedure for simulation response surface model identification. Comm. ACM 30(8):716–730.Crossref, Google Scholar
• Shao J, Tu D (1995) The Jackknife and Bootstrap (Springer, New York).Crossref, Google Scholar
• Shen H, Wan H (2009) Controlled sequential factorial design for simulation factor screening. Eur. J. Oper. Res. 198(2):511–519.Crossref, Google Scholar
• Shi W, Kleijnen JPC, Liu Z (2014) Factor screening for simulation with multiple responses: Sequential bifurcation. Eur. J. Oper. Res. 237(1):136–147.Crossref, Google Scholar
• Shi W, Shang J, Zhang Z (2016) Simulation screening and false discovery rate control for both main and interaction effects. Roeder TMK, Frazier PI, Szechtman RE, Zhou TH, Chick SE, eds. Proc. 2016 Winter Simulation Conf. (Institute of Electrical and Electronics Engineers, Washington, DC), 512–521.Google Scholar
• Storey JD (2002) A direct approach to false discovery rates. J. Roy. Stat. Soc. Ser. B.: Stat. Methodology 64(3):479–498.Crossref, Google Scholar
• Storey JD (2003) The positive false discovery rate: A Bayesian interpretation and the q-value. Ann. Statist. 31(6):2013–2035.Crossref, Google Scholar
• Sun L, Craiu RV, Paterson AD, Bull SB (2006) Stratified false discovery control for large-scale hypothesis testing with application to genome-wide association studies. Genetic Epidemiology 30(6):519–530.Crossref, Google Scholar
• TOP500.org (2018) China races ahead in TOP500 supercomputer list, ending US supremacy. Accessed July 1, 2016, https://www.top500.org/news/china-races-ahead-in-top500-supercomputer-list-ending-us-supremacy/.Google Scholar
• Tsagris M, Beneki C, Hassani H (2014) On the folded normal distribution. Mathematics 2(1):12–28.Crossref, Google Scholar
• Wan H, Ankenman BE, Nelson BL (2006) Controlled sequential bifurcation: A new factor-screening method for discrete-event simulation. Oper. Res. 54(4):743–755.Link, Google Scholar
• Wan H, Ankenman BE, Nelson BL (2010) Improving the efficiency and efficacy of controlled sequential bifurcation for simulation factor screening. INFORMS J. Comput. 22(3):482–492.Link, Google Scholar
• Wang W, Wan H (2014) Sequential procedures for multiple responses factor screening. Tolk A, Diallo SD, Ryzhov IO, Yilmaz L, Buckley S, Miller JA, eds. Proc. 2014 Winter Simulation Conf. (IEEE, Piscataway, NJ), 745–756.Crossref, Google Scholar
• Wedge DC, Rowe W, Kell DB, Knowles J (2009) In silico modelling of directed evolution: Implications for experimental design and stepwise evolution. J. Theoret. Biol. 257(1):131–141.Crossref, Google Scholar
• Woods DC, Lewis SM (2016) Design of experiments for screening. Ghanem R, Higdon D, Owhadi H, eds. Handbook of Uncertainty Quantification (Springer, New York), 1–43.Google Scholar
• Wu CFJ, Hamada M (2009) Experiments: Planning, Analysis and Optimization, 2nd ed. (Wiley, Hoboken, NJ).Google Scholar
• Wu X, Zhu X, Wu GQ, Ding W (2014) Data mining with big data. IEEE Trans. Knowledge Data Engrg. 26(1):97–107.Crossref, Google Scholar
• Xing D, Wan H, Zhu Y, Sanchez SM, Kaymal T (2013) Simulation screening experiments using LASSO-optimal supersaturated design and analysis: A maritime operations application. Pasupathy R, Kim SH, Tolk A, Hill R, Kuhl ME, eds. Proc. 2013 Winter Simulation Conf. (IEEE, Piscataway, NJ), 497–508.Crossref, Google Scholar