Quantifying Input Uncertainty via Simulation Confidence Intervals

We consider the problem of deriving confidence intervals for the mean response of a system that is represented by a stochastic simulation whose parametric input models have been estimated from “real-world” data. As opposed to standard simulation confidence intervals, we provide confidence intervals that account for uncertainty about the input model parameters; our method is appropriate when enough simulation effort can be expended to make simulation-estimation error relatively small. To achieve this we introduce metamodel-assisted bootstrapping that propagates input variability through to the simulation response via an equation-based model rather than by simulating. We develop a metamodel strategy and associated experiment design method that avoid the need for low-order approximation to the response and that minimizes the impact of intrinsic (simulation) error on confidence level accuracy. Asymptotic analysis and empirical tests over a wide range of simulation effort show that confidence intervals obtained via metamodel-assisted bootstrapping achieve the desired coverage.