Optimization-Based Calibration of Simulation Input Models

References

• Arcones MA, Gine E (1993) Limit theorems for U-processes. Ann. Probab. 21(3):1494–1542.Crossref, Google Scholar
• Avellaneda M, Buff R, Friedman C, Grandechamp N, Kruk L, Newman J (2001) Weighted Monte Carlo: A new technique for calibrating asset-pricing models. Internat. J. Theoret. Appl. Finance 4(1):91–119.Crossref, Google Scholar
• Balci O, Sargent RG (1982) Some examples of simulation model validation using hypothesis testing. Proc. 14th Winter Simulation Conf., vol. 2 (IEEE Press, Piscataway, NJ), 621–629.Google Scholar
• Banks J, Carson J, Nelson B, Nicol D (2009) Discrete-Event System Simulation, 5th ed. (Prentice Hall, Englewood Cliffs, NJ).Google Scholar
• Barton RR (2012) Tutorial: Input uncertainty in outout analysis. Proc. 2012 Winter Simulation Conf. (IEEE Press, Piscataway, NJ), 1–12.Crossref, Google Scholar
• Barton RR, Schruben LW (2001) Resampling methods for input modeling. Proc. 2001 Winter Simulation Conf., vol. 1 (IEEE Press, Piscataway, NJ), 372–378.Crossref, Google Scholar
• Barton RR, Nelson BL, Xie W (2013) Quantifying input uncertainty via simulation confidence intervals. INFORMS J. Comput. 26(1):74–87.Link, Google Scholar
• Basawa I, Bhat U, Zhou J (2008) Parameter estimation using partial information with applications to queueing and related models. Statist. Probab. Lett. 78(12):1375–1383.Crossref, Google Scholar
• Basawa IV, Bhat UN, Lund R (1996) Maximum likelihood estimation for single server queues from waiting time data. Queueing Systems 24(1–4):155–167.Crossref, Google Scholar
• Bayraksan G, Love DK (2015) Data-driven stochastic programming using phi-divergences. Aleman DM, Thiele AC, Smith JC, eds. The Operations Research Revolution, TutORials in Operations Research (INFORMS, Catonsville, MD), 1–19.Google Scholar
• Beck A, Teboulle M (2003) Mirror descent and nonlinear projected subgradient methods for convex optimization. Oper. Res. Lett. 31(3):167–175.Crossref, Google Scholar
• Ben-Tal A, El Ghaoui L, Nemirovski A (2009) Robust Optimization (Princeton University Press, Princeton, NJ).Crossref, Google Scholar
• Ben-Tal A, Den Hertog D, De Waegenaere A, Melenberg B, Rennen G (2013) Robust solutions of optimization problems affected by uncertain probabilities. Management Sci. 59(2):341–357.Link, Google Scholar
• Benveniste A, Métivier M, Priouret P (2012) Adaptive Algorithms and Stochastic Approximations, Applications of Mathematics, vol. 22 (Springer Science & Business Media, Berlin).Google Scholar
• Bertsekas DP (1999) Nonlinear Programming (Athena Scientific, Nashua, NH).Google Scholar
• Bertsimas D, Natarajan K (2007) A semidefinite optimization approach to the steady-state analysis of queueing systems. Queueing Systems 56(1):27–39.Crossref, Google Scholar
• Bertsimas D, Popescu I (2005) Optimal inequalities in probability theory: A convex optimization approach. SIAM J. Optim. 15(3):780–804.Crossref, Google Scholar
• Bertsimas D, Brown DB, Caramanis C (2011) Theory and applications of robust optimization. SIAM Rev. 53(3):464–501.Crossref, Google Scholar
• Bertsimas D, Gupta V, Kallus N (2018) Robust sample average approximation. Math. Programming 171(1/2):217–282.Crossref, Google Scholar
• Bingham N, Pitts SM (1999) Non-parametric estimation for the M/G/∞ queue. Ann. Inst. Statist. Math. 51(1):71–97.Crossref, Google Scholar
• Blanchet J, Murthy KR (2019) Quantifying distributional model risk via optimal transport. Math. Oper. Res. 44(2):565–600.Link, Google Scholar
• Blanchet J, Kang Y, Murthy K (2016) Robust Wasserstein profile inference and applications to machine learning. Working paper, Columbia University, New York.Google Scholar
• Blum JR (1954) Multidimensional stochastic approximation methods. Ann. Math. Statist. 25(4):737–744.Crossref, Google Scholar
• Bottou L (2009) Online learning and stochastic approximations. Saad D, ed. On-line Learning in Neural Networks (Cambridge University Press, New York), 9–42.Google Scholar
• Broadie M, Cicek D, Zeevi A (2011) General bounds and finite-time improvement for the Kiefer-Wolfowitz stochastic approximation algorithm. Oper. Res. 59(5):1211–1224.Link, Google Scholar
• Cheng RC, Holland W (1998) Two-point methods for assessing variability in simulation output. J. Statist. Comput. Simulation 60(3):183–205.Crossref, Google Scholar
• Cheng RC, Holland W (2004) Calculation of confidence intervals for simulation output. ACM Trans. Model. Comput. Simulation (TOMACS) 14(4):344–362.Crossref, Google Scholar
• Chick SE (2001) Input distribution selection for simulation experiments: Accounting for input uncertainty. Oper. Res. 49(5):744–758.Link, Google Scholar
• Chick SE, Ng SH (2002) Simulation input analysis: Joint criterion for factor identification and parameter estimation. Proc. 34th Winter Simulation Conf. (IEEE Press, Piscataway, NJ), 400–406.Crossref, Google Scholar
• Cooper RB (1981) Introduction to Queueing Theory, 2nd ed. (North Holland, New York).Google Scholar
• Csiszár I (1991) Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems. Ann. Statist. 19(4):2032–2066.Crossref, Google Scholar
• Currin C, Mitchell T, Morris M, Ylvisaker D (1991) Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments. J. Amer. Statist. Assoc. 86(416):953–963.Crossref, Google Scholar
• Daley D, Servi L (1998) Moment estimation of customer loss rates from transactional data. Internat. J. Stochastic Anal. 11(3):301–310.Crossref, Google Scholar
• Dang CD, Lan G (2015) Stochastic block mirror descent methods for nonsmooth and stochastic optimization. SIAM J. Optim. 25(2):856–881.Crossref, Google Scholar
• Delage E, Ye Y (2010) Distributionally robust optimization under moment uncertainty with application to data-driven problems. Oper. Res. 58(3):595–612.Link, Google Scholar
• Donoho DL, Johnstone IM, Hoch JC, Stern AS (1992) Maximum entropy and the nearly black object. J. Roy. Statist. Soc.: Ser. B Methodology 54:41–81.Google Scholar
• Duchi J, Glynn P, Namkoong H (2019) Statistics of robust optimization: A generalized empirical likelihood approach. Math. Oper. Res. Forthcoming.Google Scholar
• Esfahani PM, Kuhn D (2017) Data-driven distributionally robust optimization using the Wasserstein metric: Performance guarantees and tractable reformulations. Math. Programming 171(1/2):1–52.Google Scholar
• Fan W, Hong LJ, Zhang X (2013) Robust selection of the best. Proc. 2013 Winter Simulation Conf. (IEEE Press, Piscataway, NJ), 868–876.Crossref, Google Scholar
• Fearnhead P (2004) Filtering recursions for calculating likelihoods for queues based on inter-departure time data. Statist. Comput. 14(3):261–266.Crossref, Google Scholar
• Feng H, Dube P, Zhang L (2014) Estimating life-time distribution by observing population continuously. Performance Evaluation 79:182–197.Crossref, Google Scholar
• Frey JC, Kaplan EH (2010) Queue inference from periodic reporting data. Oper. Res. Lett. 38(5):420–426.Crossref, Google Scholar
• Gao R, Kleywegt AJ (2016) Distributionally robust stochastic optimization with Wasserstein distance. Working paper, Georgia Institute of Technology, Atlanta.Google Scholar
• Ghadimi S, Lan G (2013) Stochastic first-and zeroth-order methods for nonconvex stochastic programming. SIAM J. Optim. 23(4):2341–2368.Crossref, Google Scholar
• Ghadimi S, Lan G (2015) Accelerated gradient methods for nonconvex nonlinear and stochastic programming. Math. Programming 156(1/2):59–99.Google Scholar
• Ghadimi S, Lan G, Zhang H (2016) Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization. Math. Programming 155(1/2):267–305.Crossref, Google Scholar
• Ghosh S, Lam H (2015) Mirror descent stochastic approximation for computing worst-case stochastic input models. Proc. 2015 Winter Simulation Conf. (IEEE Press, Piscataway, NJ), 425–436.Crossref, Google Scholar
• Ghosh S, Lam H (2019) Robust analysis in stochastic simulation: Computation and performance guarantees. Oper. Res. 67(1):232–249.Link, Google Scholar
• Glasserman P, Xu X (2013) Robust portfolio control with stochastic factor dynamics. Oper. Res. 61(4):874–893.Link, Google Scholar
• Glasserman P, Xu X (2014) Robust risk measurement and model risk. Quant. Finance 14(1):29–58.Crossref, Google Scholar
• Glasserman P, Yang L (2018) Bounding wrong-way risk in CVA calculation. Math. Finance 28(1):268–305.Crossref, Google Scholar
• Glasserman P, Yu B (2005) Large sample properties of weighted Monte Carlo estimators. Oper. Res. 53(2):298–312.Link, Google Scholar
• Goeva A, Lam H, Zhang B (2014) Reconstructing input models via simulation optimization. Proc. 2014 Winter Simulation Conf. (IEEE Press, Piscataway, NJ), 698–709.Crossref, Google Scholar
• Goh J, Sim M (2010) Distributionally robust optimization and its tractable approximations. Oper. Res. 58(4-part-1):902–917.Link, Google Scholar
• Gupta V (2019) Near-optimal ambiguity sets for distributionally robust optimization. Management Sci., ePub ahead of print March 18, https://pubsonline.informs.org/doi/10.1287/mnsc.2018.3140.Link, Google Scholar
• Hall P, Park J (2004) Nonparametric inference about service time distribution from indirect measurements. J. Roy. Statist. Soc.: Ser. B Statist. Methodology 66(4):861–875.Crossref, Google Scholar
• Hanasusanto GA, Roitch V, Kuhn D, Wiesemann W (2017) Ambiguous joint chance constraints under mean and dispersion information. Oper. Res. 65(3):751–767.Link, Google Scholar
• Hansen LP, Sargent TJ (2008) Robustness (Princeton University Press, Princeton, NJ).Crossref, Google Scholar
• Heckmüller S, Wolfinger BE (2009) Reconstructing arrival processes to G/D/1 queueing systems and tandem networks. Proc. Internat. Sympos. Performance Evaluation Comput. Telecomm. Systems, 2009 (SPECTS 2009), vol. 41 (IEEE Press, Piscataway, NJ), 361–368.Google Scholar
• Hu Z, Cao J, Hong LJ (2012) Robust simulation of global warming policies using the dice model. Management Sci. 58(12):2190–2206.Link, Google Scholar
• Iyengar GN (2005) Robust dynamic programming. Math. Oper. Res. 30(2):257–280.Link, Google Scholar
• Jain A, Lim A, Shanthikumar J (2010) On the optimality of threshold control in queues with model uncertainty. Queueing Systems 65(2):157–174.Crossref, Google Scholar
• Kelton WD, Law AM (2000) Simulation Modeling and Analysis (McGraw Hill, Boston).Google Scholar
• Kennedy MC, O’Hagan A (2001) Bayesian calibration of computer models. J. Roy. Statist. Soc.: Ser. B. Statist. Methodology 63(3):425–464.Crossref, Google Scholar
• Kim YB, Park J (2008) New approaches for inference of unobservable queues. Proc. 40th Conf. Winter Simulation (IEEE Press, Piscataway, NJ), 2820–2825.Crossref, Google Scholar
• Kleijnen JP (1995) Verification and validation of simulation models. Eur. J. Oper. Res. 82(1):145–162.Crossref, Google Scholar
• Kraan B, Bedford T (2005) Probabilistic inversion of expert judgments in the quantification of model uncertainty. Management Sci. 51(6):995–1006.Link, Google Scholar
• Lam H (2016) Robust sensitivity analysis for stochastic systems. Math. Oper. Res. 41(4):1248–1275.Link, Google Scholar
• Lam H (2018) Sensitivity to serial dependency of input processes: A robust approach. Management Sci. 64(3):1311–1327.Link, Google Scholar
• Lam H (2019) Recovering best statistical guarantees via the empirical divergence-based distributionally robust optimization. Oper. Res. Forthcoming.Link, Google Scholar
• Lam H, Mottet C (2017) Tail analysis without parametric models: A worst-case perspective. Oper. Res. 65(6):1696–1711.Link, Google Scholar
• Lam H, Zhou E (2017) The empirical likelihood approach to quantifying uncertainty in sample average approximation. Oper. Res. Lett. 45(4):301–307.Crossref, Google Scholar
• Lan G, Zhou Z (2017) Algorithms for stochastic optimization with expectation constraints. Working paper, Georgia Institute of Technology, Atlanta.Google Scholar
• Larson RC (1990) The queue inference engine: Deducing queue statistics from transactional data. Management Sci. 36(5):586–601.Link, Google Scholar
• Lehmann EL, Romano JP (2006) Testing Statistical Hypotheses (Springer Science & Business Media, New York).Google Scholar
• Li B, Jiang R, Mathieu JL (2019) Ambiguous risk constraints with moment and unimodality information. Math. Programming 173(1/2):151–192.Crossref, Google Scholar
• Lim AEB, Shanthikumar JG (2007) Relative entropy, exponential utility, and robust dynamic pricing. Oper. Res. 55(2):198–214.Link, Google Scholar
• Mandelbaum A, Zeltyn S (1998) Estimating characteristics of queueing networks using transactional data. Queueing Systems 29(1):75–127.Crossref, Google Scholar
• Marjoram P, Molitor J, Plagnol V, Tavaré S (2003) Markov chain Monte Carlo without likelihoods. Proc. Natl. Acad. Sci. USA 100(26):15324–15328.Crossref, Google Scholar
• Moulines E, Roueff F, Souloumiac A, Trigano T (2007) Nonparametric inference of photon energy distribution from indirect measurement. Bernoulli 13(2):365–388.Crossref, Google Scholar
• Nelson B (2016) ‘Some tactical problems in digital simulation’ for the next 10 years. J. Simulation 10(1):2–11.Crossref, Google Scholar
• Nemirovski A, Juditsky A, Lan G, Shapiro A (2009) Robust stochastic approximation approach to stochastic programming. SIAM J. Optim. 19(4):1574–1609.Crossref, Google Scholar
• Park J, Kim YB, Willemain TR (2011) Analysis of an unobservable queue using arrival and departure times. Comput. Indust. Engrg. 61(3):842–847.Crossref, Google Scholar
• Petersen I, James M, Dupuis P (2000) Minimax optimal control of stochastic uncertain systems with relative entropy constraints. IEEE Trans. Automatic Control 45(3):398–412.Crossref, Google Scholar
• Pickands J, III, Stine RA (1997) Estimation for an M/G/∞queue with incomplete information. Biometrika 84(2):295–308.Crossref, Google Scholar
• Popescu I (2005) A semidefinite programming approach to optimal-moment bounds for convex classes of distributions. Math. Oper. Res. 30(3):632–657.Link, Google Scholar
• Ross JV, Taimre T, Pollett PK (2007) Estimation for queues from queue length data. Queueing Systems 55(2):131–138.Crossref, Google Scholar
• Ryzhov IO, Defourny B, Powell WB (2012) Ranking and selection meets robust optimization. Proc. Winter Simulation Conf. (IEEE Press, Piscataway, NJ), 532–542.Crossref, Google Scholar
• Santner TJ, Williams BJ, Notz WI (2013) The Design and Analysis of Computer Experiments (Springer Science & Business Media, New York).Google Scholar
• Sargent RG (2005) Verification and validation of simulation models. Proc. 37th Winter Simulation Conf. (IEEE Press, Piscataway, NJ), 130–143.Crossref, Google Scholar
• Schruben LW (1980) Establishing the credibility of simulations. Simulation 34(3):101–105.Crossref, Google Scholar
• Serfling RJ (2009) Approximation Theorems of Mathematical Statistics, Wiley Series in Probability and Statistics, vol. 162 (John Wiley & Sons, New York).Google Scholar
• Shafieezadeh-Abadeh S, Esfahani PM, Kuhn D (2015) Distributionally robust logistic regression. Proc. Adv. Neural Inform. Processing Systems (Curran Associates, Inc., Red Hook, NY), 1576–1584.Google Scholar
• Shirangi MG (2014) History matching production data and uncertainty assessment with an efficient TSVD parameterization algorithm. J. Petroleum Sci. Engrg. 113:54–71.Crossref, Google Scholar
• Smith JE (1995) Generalized Chebyshev inequalities: Theory and applications in decision analysis. Oper. Res. 43(5):807–825.Link, Google Scholar
• Song E, Nelson BL, Pegden CD (2014) Advanced tutorial: Input uncertainty quantification. Proc. 2014 Winter Simulation Conf. (IEEE Press, Piscataway, NJ), 162–176.Crossref, Google Scholar
• Tarantola A (2005) Inverse Problem Theory and Methods for Model Parameter Estimation (SIAM, Philadelphia).Crossref, Google Scholar
• van der Vaart AW, Wellner JA (1996) Weak Convergence and Empirical Processes (Springer, New York).Crossref, Google Scholar
• Wang IJ, Spall JC (2008) Stochastic optimisation with inequality constraints using simultaneous perturbations and penalty functions. Internat. J. Control 81(8):1232–1238.Crossref, Google Scholar
• Wang TY, Ke JC, Wang KH, Ho SC (2006) Maximum likelihood estimates and confidence intervals of an M/M/R queue with heterogeneous servers. Math. Methods Oper. Res. 63(2):371–384.Crossref, Google Scholar
• Whitt W (1981) Approximating a point process by a renewal process: The view through a queue, an indirect approach. Management Sci. 27(6):619–636.Link, Google Scholar
• Whitt W (1982) Approximating a point process by a renewal process, I: Two basic methods. Oper. Res. 30(1):125–147.Link, Google Scholar
• Whitt W (2012) Fitting birth-and-death queueing models to data. Statist. Probab. Lett. 82(5):998–1004.Crossref, Google Scholar
• Wiesemann W, Kuhn D, Sim M (2014) Distributionally robust convex optimization. Oper. Res. 62(6):1358–1376.Link, Google Scholar
• Wunsch C (1996) The Ocean Circulation Inverse Problem (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• Xie W, Ahmed S (2018) Distributionally robust chance constrained optimal power flow with renewables: A conic reformulation. IEEE Trans. Power Systems 33(2):1860–1867.Crossref, Google Scholar
• Xin L, Goldberg DA (2015) Distributionally robust inventory control when demand is a martingale. Working paper, University of Illinois at Urbana-Champaign, Champaign.Google Scholar
• Xu H, Mannor S (2012) Distributionally robust Markov decision processes. Math. Oper. Res. 37(2):288–300.Link, Google Scholar
• Yu H, Neely M, Wei X (2017) Online convex optimization with stochastic constraints. Proc. Adv. Neural Inform. Processing Systems (Curran Associates, Inc., Red Hook, NY), 1427–1437.Google Scholar
• Zhang Y, Shen S, Mathieu JL (2017) Distributionally robust chance-constrained optimal power flow with uncertain renewables and uncertain reserves provided by loads. IEEE Trans. Power Systems 32(2):1378–1388.Google Scholar
• Zhao C, Jiang R (2018) Distributionally robust contingency-constrained unit commitment. IEEE Trans. Power Systems 33(1):94–102.Crossref, Google Scholar
• Zouaoui F, Wilson JR (2004) Accounting for input-model and input-parameter uncertainties in simulation. IIE Trans. 36(11):1135–1151.Crossref, Google Scholar