Minimax Optimal Estimation of Stability Under Distribution Shift

Published Online:https://doi.org/10.1287/opre.2022.0658

References

  • ACCORD Study Group (2010) Effects of intensive blood-pressure control in type 2 diabetes mellitus. New Engl. J. Med. 362(17):1575–1585.CrossrefGoogle Scholar
  • Amorim E, Cançado M, Veloso A (2018) Automated essay scoring in the presence of biased ratings. Proc. 2018 Conf. North Amer. Chapter Assoc. Comput. Linguistics (Association for Computational Linguistics, Kerrville, TX), 229–237.Google Scholar
  • Artzner P, Delbaen F, Eber J-M, Heath D (1999) Coherent measures of risk. Math. Finance 9(3):203–228.CrossrefGoogle Scholar
  • Asmussen S (2000) Ruin Probabilities, Advanced Series on Statistical Science & Applied Probability, vol. 2 (World Scientific, Singapore).CrossrefGoogle Scholar
  • Asmussen S, Glynn PW (2007) Stochastic Simulation: Algorithms and Analysis (Springer, New York).CrossrefGoogle Scholar
  • Athey S (2017) Beyond prediction: Using big data for policy problems. Science 355(6324):483–485.CrossrefGoogle Scholar
  • Bachoc F, Gamboa F, Halford M, Loubes J-M, Risser L (2023) Explaining machine learning models using entropic variable projection. Inform. Inference J. IMA 12(3):1686–1715.Google Scholar
  • Bandi P, Geessink O, Manson Q, Van Dijk M, Balkenhol M, Hermsen M, Bejnordi BE, et al. (2019) From detection of individual metastases to classification of lymph node status at the patient level: The CAMELYON17 challenge. IEEE Trans. Med. Imaging 38(2):550–560.CrossrefGoogle Scholar
  • Banerjee A, Karlan D, Zinman J (2015) Six randomized evaluations of microcredit: Introduction and further steps. Amer. Econom. J. Appl. Econom. 7(1):1–21.CrossrefGoogle Scholar
  • Basu S, Sussman JB, Hayward RA (2017) Detecting heterogeneous treatment effects to guide personalized blood pressure treatment: A modeling study of randomized clinical trials. Ann. Intern. Med. 166(5):354–360.CrossrefGoogle Scholar
  • Beery S, Cole E, Gjoka A (2020) The iWildCam 2020 competition dataset. Preprint, submitted April 21, https://arxiv.org/abs/2004.10340.Google Scholar
  • Berk RH (1966) Limiting behavior of posterior distributions when the model is incorrect. Ann. Math. Statist. 37(1):51–58.CrossrefGoogle Scholar
  • Blanchet J, Murthy K (2019) Quantifying distributional model risk via optimal transport. Math. Oper. Res. 44(2):565–600.LinkGoogle Scholar
  • Blanchet J, Kang Y, Murthy K (2019) Robust Wasserstein profile inference and applications to machine learning. J. Appl. Probab. 56(3):830–857.CrossrefGoogle Scholar
  • Broderick T, Giordano R, Meager R (2023) An automatic finite-sample robustness metric: When can dropping a little data make a big difference? Preprint, submitted July 19, https://arxiv.org/abs/2011.14999.Google Scholar
  • Brown LD (1986) Fundamentals of Statistical Exponential Families (Institute of Mathematical Statistics, Hayward, CA).Google Scholar
  • Chen MS, Lara PN, Dang JH, Paterniti DA, Kelly K (2014) Twenty years post-NIH revitalization act: Enhancing minority participation in clinical trials (EMPaCT): Laying the groundwork for improving minority clinical trial accrual: Renewing the case for enhancing minority participation in cancer clinical trials. Cancer 120(57):1091–1096.CrossrefGoogle Scholar
  • Chen IY, Pierson E, Rose S, Joshi S, Ferryman K, Ghassemi M (2020) Ethical machine learning in health care. Preprint, submitted October 8, https://arxiv.org/abs/2009.10576.Google Scholar
  • Cook I (2021) Who is driving the great resignation? Harvard Bus. Rev. (September 15), https://hbr.org/2021/09/who-is-driving-the-great-resignation.Google Scholar
  • Cornfield J, Haenszel W, Hammond EC, Lilienfeld AM, Shimkin MB, Wynder EL (1959) Smoking and lung cancer: Recent evidence and a discussion of some questions. J. Natl. Cancer Inst. 22(1):173–203.Google Scholar
  • Cruces G, Galiani S (2007) Fertility and female labor supply in Latin America: New causal evidence. Labour Econom. 14(3):565–573.CrossrefGoogle Scholar
  • Csiszar I (1984) Sanov property, generalized I-projection and a conditional limit theorem. Ann. Probab. 12(3):768–793.CrossrefGoogle Scholar
  • Csörgő S, Teugels JL (1990) Empirical laplace transform and approximation of compound distributions. J. Appl. Probab. 27(1):88–101.CrossrefGoogle Scholar
  • Currie J, Fahr J (2005) Medicaid managed care: Effects on children’s Medicaid coverage and utilization. J. Public Econom. 89(1):85–108.CrossrefGoogle Scholar
  • Currie J, Gruber J (1996) Health insurance eligibility, utilization of medical care, and child health. Quart. J. Econom. 111(2):431–466.CrossrefGoogle Scholar
  • Dai JG, Gluzman M (2022) Queueing network controls via deep reinforcement learning. Stochastic Systems 12(1):30–67.LinkGoogle Scholar
  • D’Amour A, Heller K, Moldovan D, Adlam B, Alipanahi B, Beutel A, Chen C, et al. (2022) Underspecification presents challenges for credibility in modern machine learning. J. Machine Learn. Res. 23(226):1–61.Google Scholar
  • Dehejia R, Pop-Eleches C, Samii C (2021) From local to global: External validity in a fertility natural experiment. J. Bus. Econom. Statist. 39(1):217–243.CrossrefGoogle Scholar
  • Delbaen F (2002) Coherent risk measures on general probability spaces. Sandmann K, Schönbucher PJ, eds. Advances in Finance and Stochastics (Springer, Berlin, Heidelberg), 1–37.CrossrefGoogle Scholar
  • Dembo A, Zeitouni O (1998) Large Deviations Techniques and Applications (Springer-Verlag, Berlin, Heidelberg).CrossrefGoogle Scholar
  • Deuschel J-D, Stroock DW (1989) Large Deviations, Pure and Applied Mathematics, vol. 137 (Academic Press, Boston).Google Scholar
  • Donsker MD, Varadhan SRS (1976) Asymptotic evaluation of certain Markov process expectations for large time-III. Commun. Pure Appl. Math. 29(4):389–461.CrossrefGoogle Scholar
  • Duchi JC, Namkoong H (2021) Learning models with uniform performance via distributionally robust optimization. Ann. Statist. 49(3):1378–1406.CrossrefGoogle Scholar
  • Duchi JC, Glynn PW, Namkoong H (2021) Statistics of robust optimization: A generalized empirical likelihood approach. Math. Oper. Res. 46(3):946–969.LinkGoogle Scholar
  • Duchi J, Hashimoto T, Namkoong H (2023) Distributionally robust losses for latent covariate mixtures. Oper. Res. 71(2):649–664.LinkGoogle Scholar
  • Duffy K, Metcalfe AP (2005) The large deviations of estimating rate functions. J. Appl. Probab. 42(1):267–274.CrossrefGoogle Scholar
  • Duffy KR, Williamson BD (2015) Estimating large deviation rate functions. Preprint, submitted November 7, https://arxiv.org/abs/1511.02295v1.Google Scholar
  • Eick SG, Massey WA, Whitt W (1993) Mt/G/∞ queues with sinusoidal arrival rates. Management Sci. 39(2):241–252.LinkGoogle Scholar
  • Ellis R (2007) Entropy, Large Deviations, and Statistical Mechanics (Springer, Berlin, Heidelberg).Google Scholar
  • Feuerverger A (1989) On the empirical saddlepoint approximation. Biometrika 76(3):457–464.CrossrefGoogle Scholar
  • Feuerverger A, Mureika RA (1977) The empirical characteristic function and its applications. Ann. Statist. 5(1):88–97.CrossrefGoogle Scholar
  • Freund D, Hopkins SB (2023) Towards practical robustness auditing for linear regression. Preprint, submitted July 30, https://arxiv.org/abs/2307.16315.Google Scholar
  • Gao R, Kleywegt AJ (2023) Distributionally robust stochastic optimization with Wasserstein distance. Math. Oper. Res. 48(2):603–655.LinkGoogle Scholar
  • Garg N, Nazerzadeh H (2022) Driver surge pricing. Management Sci. 68(5):3219–3235.LinkGoogle Scholar
  • Gertler P, Shah M, Alzua ML, Cameron L, Martinez S, Patil S (2015) How does health promotion work? Evidence from the dirty business of eliminating open defecation. NBER Working Paper No. 20997, National Bureau of Economic Research, Cambridge, MA.Google Scholar
  • Ghosh S, Lam H (2019) Robust analysis in stochastic simulation: Computation and performance guarantees. Oper. Res. 67(1):232–249.LinkGoogle Scholar
  • Glasserman P (2004) Monte Carlo Methods in Financial Engineering, Stochastic Modelling and Applied Probability, vol. 53 (Springer, New York).Google Scholar
  • Glasserman P, Xu X (2014) Robust risk measurement and model risk. Quant. Finance 14(1):29–58.CrossrefGoogle Scholar
  • Glasserman P, Yang L (2018) Bounding wrong-way risk in CVA calculation. Math. Finance 28(1):268–305.CrossrefGoogle Scholar
  • Glynn PW, Zheng Z (2019) Estimation and inference for non-stationary arrival models with a linear trend. Winter Simul. Conf. WSC (IEEE, Piscataway, NJ), 3764–3773.Google Scholar
  • Goeva A, Lam H, Qian H, Zhang B (2019) Optimization-based calibration of simulation input models. Oper. Res. 67(5):1362–1382.LinkGoogle Scholar
  • Green L, Kolesar P (1989) Testing the validity of a queueing model of police patrol. Management Sci. 35(2):127–148.LinkGoogle Scholar
  • Green L, Kolesar P, Svoronos A (1991) Some effects of nonstationarity on multiserver Markovian queueing systems. Oper. Res. 39(3):502–511.LinkGoogle Scholar
  • Grinsztajn L, Oyallon E, Varoquaux G (2022) Why do tree-based models still outperform deep learning on tabular data? Proc. 36th Internat. Conf. Neural Inform. Processing Systems (Curran Associates, Red Hook, NY), 507–520.Google Scholar
  • Gu S, Holly E, Lillicrap T, Levine S (2017) Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. Internat. Conf. Robotics Automation (IEEE, Piscataway, NJ), 3389–3396.Google Scholar
  • Gupta S, Rothenhäusler D (2021) The s-value: Evaluating stability with respect to distributional shifts. Preprint, submitted May 7, https://arxiv.org/abs/2105.03067v1.Google Scholar
  • Hall P, Teugels JL, Vanmarcke A (1992) The abscissa of convergence of the Laplace transform. J. Appl. Probab. 29(2):353–362.CrossrefGoogle Scholar
  • Hand DJ (2006) Classifier technology and the illusion of progress. Statist. Sci. 21(1):1–14.CrossrefGoogle Scholar
  • Hansen LP, Sargent TJ (2001) Robust control and model uncertainty. Amer. Econom. Rev. 91(2):60–66.CrossrefGoogle Scholar
  • Harrison JM, Wein LM (1989) Scheduling networks of queues: Heavy traffic analysis of a simple open network. Queueing Systems 5(2):265–280.CrossrefGoogle Scholar
  • Harrison JM, Zeevi A (2004) Dynamic scheduling of a multiclass queue in the Halfin-Whitt heavy traffic regime. Oper. Res. 52(2):243–257.LinkGoogle Scholar
  • Hastie T, Tibshirani R, Friedman J (2009) The Elements of Statistical Learning, 2nd ed. (Springer, New York).CrossrefGoogle Scholar
  • Henderson P, Islam R, Bachman P, Pineau J, Precup D, Meger D (2018) Deep reinforcement learning that matters. Thirty-Second AAAI Conf. Artificial Intelligence (AAAI Press, Washington, DC), 3207–3214.Google Scholar
  • Homma T, Saltelli A (1996) Importance measures in global sensitivity analysis of nonlinear models. Reliability Engrg. System Safety 52(1):1–17.CrossrefGoogle Scholar
  • Hu Y, Chan CW, Dong J (2025) Prediction-driven surge planning with application to emergency department nurse staffing. Management Sci. 71(3):2079–2126.Google Scholar
  • Huber PJ (1981) Robust Statistics (John Wiley and Sons, New York).CrossrefGoogle Scholar
  • Huber PJ, Ronchetti EM (2009) Robust Statistics, 2nd ed. (John Wiley and Sons, New York).CrossrefGoogle Scholar
  • Ioannidis JP (2005) Why most published research findings are false. PLoS Med. 2(8):e124.CrossrefGoogle Scholar
  • Jeong S, Namkoong H (2020a) Robust causal inference under covariate shift via worst-case subpopulation treatment effect. Proc. Thirty-Third Annu. Conf. Comput. Learn. Theory (PMLR, Graz, Austria), 2079–2084.Google Scholar
  • Jeong S, Namkoong H (2020b) Assessing external validity over worst-case subpopulations. Preprint, submitted July 5, https://arxiv.org/abs/2007.02411v1.Google Scholar
  • Jiang WX, Nelson BL, Hong LJ (2019) Estimating sensitivity to input model variance. Proc. 2019 Winter Simul. Conf. (IEEE, Piscataway, NJ), 3705–3716.Google Scholar
  • Kahende J, Malarcher A, England L, Zhang L, Mowery P, Xu X, Sevilimedu V, Rolle I (2017) Utilization of smoking cessation medication benefits among Medicaid fee-for-service enrollees 1999–2008. PLoS One 12(2):e0170381.CrossrefGoogle Scholar
  • Kallus N, Zhou A (2018) Confounding-robust policy improvement. Adv. Neural Inform. Processing Systems 31:9269–9279.Google Scholar
  • Kleinberg J, Ludwig J, Mullainathan S, Obermeyer Z (2015) Prediction policy problems. Amer. Econom. Rev. 105(5):491–495.CrossrefGoogle Scholar
  • Koh PW, Sagawa S, Marklund H, Xie SM, Zhang M, Balsubramani A, Hu W, et al. (2020) WILDS: A benchmark of in-the-wild distribution shifts. Preprint, submitted December 14, https://arxiv.org/abs/2012.07421.Google Scholar
  • Koopman BO (1972) Air-terminal queues under time-dependent conditions. Oper. Res. 20(6):1089–1114.LinkGoogle Scholar
  • Kuhn D, Esfahani PM, Nguyen VA, Shafieezadeh-Abadeh S (2019) Wasserstein distributionally robust optimization: Theory and applications in machine learning. Operations Research & Management Science in the Age of Analytics (INFORMS, Catonsville, MD), 130–166.LinkGoogle Scholar
  • Lam H (2016a) Robust sensitivity analysis for stochastic systems. Math. Oper. Res. 41(4):1248–1275.LinkGoogle Scholar
  • Lam H (2016b) Advanced tutorial: Input uncertainty and robust analysis in stochastic simulation. Proc. 2016 Winter Simul. Conf. (IEEE, Piscataway, NJ), 178–192.Google Scholar
  • Lam H (2018) Sensitivity to serial dependency of input processes: A robust approach. Management Sci. 64(3):1311–1327.LinkGoogle Scholar
  • Lam H, Qian H (2019) Optimization-based quantification of simulation input uncertainty via empirical likelihood. Preprint, submitted February 13, https://arxiv.org/abs/1707.05917.Google Scholar
  • Lam H, Zhou E (2017) The empirical likelihood approach to quantifying uncertainty in sample average approximation. Oper. Res. Lett. 45(4):301–307.CrossrefGoogle Scholar
  • Le Cam L (1973) Convergence of estimates under dimensionality restrictions. Ann. Statist. 1(1):38–53.Google Scholar
  • Le Cam L (1986) Asymptotic Methods in Statistical Decision Theory (Springer, New York).CrossrefGoogle Scholar
  • Leek JT, Scharpf RB, Bravo HC, Simcha D, Langmead B, Johnson WE, Geman D, Baggerly K, Irizarry RA (2010) Tackling the widespread and critical impact of batch effects in high-throughput data. Nat. Rev. Genet. 11(10):733–739.CrossrefGoogle Scholar
  • Lemaître P, Sergienko E, Arnaud A, Bousquet N, Gamboa F, Iooss B (2015) Density modification-based reliability sensitivity analysis. J. Statist. Comput. Simul. 85(6):1200–1223.CrossrefGoogle Scholar
  • Levy D, Carmon Y, Duchi JC, Sidford A (2020) Large-scale methods for distributionally robust optimization. Adv. Neural Inform. Processing Systems 33:8847–8860.Google Scholar
  • Li M, Namkoong H, Xia S (2021) Evaluating model performance under worst-case subpopulations. Adv. Neural Inform. Processing Systems 34:17325–17334.Google Scholar
  • Li Q, Brown JB, Huang H, Bickel PJ (2011) Measuring reproducibility of high-throughput experiments. Ann. Appl. Stat. 5(3):1752–1779.CrossrefGoogle Scholar
  • Lipton BJ, Decker SL (2015) The effect of health insurance coverage on medical care utilization and health outcomes: Evidence from Medicaid adult vision benefits. J. Health Econom. 44:320–332.CrossrefGoogle Scholar
  • Long DZ, Sim M, Zhou M (2023) Robust satisficing. Oper. Res. 71(1):61–82.LinkGoogle Scholar
  • Mandelbaum A, Stolyar AL (2004) Scheduling flexible servers with convex delay costs: Heavy-traffic optimality of the generalized cμ-rule. Oper. Res. 52(6):836–855.LinkGoogle Scholar
  • Manski CF (2013) Public Policy in an Uncertain World (Harvard University Press, Cambridge, MA).CrossrefGoogle Scholar
  • Maronna R, Martin D, Yohai V (2006) Robust Statistics: Theory and Methods, Wiley Series in Probability and Statistics (Wiley, Hoboken, NJ).CrossrefGoogle Scholar
  • Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller M (2013) Playing Atari with deep reinforcement learning. Preprint, submitted December 19, https://arxiv.org/abs/1312.5602.Google Scholar
  • Moitra A, Rohatgi D (2023) Provably auditing ordinary least squares in low dimensions. Proc. Eleventh Internat. Conf. Learn. Representations.Google Scholar
  • Murdoch WJ, Singh C, Kumbier K, Abbasi-Asl R, Yu B (2019) Definitions, methods, and applications in interpretable machine learning. Proc. Natl. Acad. Sci. USA 116(44):22071–22080.CrossrefGoogle Scholar
  • Owen AB (2014) Sobol’ indices and Shapley value. SIAM/ASA J. Uncertainty Quantification. 2(1):245–251.CrossrefGoogle Scholar
  • Papadimitriou CH, Tsitsiklis JN (1999) The complexity of optimal queuing network control. Math. Oper. Res. 24(2):293–305.LinkGoogle Scholar
  • Rahimian H, Mehrotra S (2019) Distributionally robust optimization: A review. Preprint, submitted August 13, https://arxiv.org/abs/1908.05659.Google Scholar
  • Rockafellar RT (2007) Coherent approaches to risk in optimization under uncertainty. Tutorials Oper. Res. 3:38–61.Google Scholar
  • Rockafellar RT, Uryasev S (2000) Optimization of conditional value-at-risk. J. Risk 2:21–42.CrossrefGoogle Scholar
  • Rockafellar RT, Uryasev S, Zabarankin M (2006) Generalized deviations in risk analysis. Finance Stochastics 10(1):51–74.CrossrefGoogle Scholar
  • Rohwer CM, Angeletti F, Touchette H (2015) Convergence of large-deviation estimators. Phys. Rev. E 92(5):052104.CrossrefGoogle Scholar
  • Rosenbaum PR (2002) Observational studies. Observational Studies (Springer, New York), 1–17.CrossrefGoogle Scholar
  • Rosenbaum PR (2010) Design of Observational Studies, Springer Series in Statistics (Springer, Cham, Switzerland).CrossrefGoogle Scholar
  • Rosenbaum PR (2011) A new u-statistic with superior design sensitivity in matched observational studies. Biometrics 67(3):1017–1027.CrossrefGoogle Scholar
  • Rosenzweig MR, Udry C (2020) External validity in a stochastic world: Evidence from low-income countries. Rev. Econom. Stud. 87(1):343–381.CrossrefGoogle Scholar
  • Ruszczyński A, Shapiro A (2006) Optimization of convex risk functions. Math. Oper Res. 31(3):433–452.LinkGoogle Scholar
  • Saenko K, Kulis B, Fritz M, Darrell T (2010) Adapting visual category models to new domains. ECCV 2010 Proc. Eur. Conf. Comput. Vision (Springer, Berlin, Heidelberg), 213–226.Google Scholar
  • Saltelli A, Ratto M, Andres T, Campolongo F, Cariboni J, Gatelli D, Saisana M, Tarantola S (2008) Global Sensitivity Analysis: The Primer (John Wiley & Sons, Hoboken, NJ).Google Scholar
  • Shapiro A, Dentcheva D, Ruszczyński A (2009) Lectures on Stochastic Programming: Modeling and Theory (SIAM and Mathematical Programming Society, Philadelphia).CrossrefGoogle Scholar
  • Silver D, Schrittwieser J, Simonyan K, Antonoglou I, Huang A, Guez A, Hubert T, Baker L, Lai M, Bolton A (2017) Mastering the game of Go without human knowledge. Nature 550(7676):354.CrossrefGoogle Scholar
  • Song E, Nelson BL, Pegden CD (2014) Advanced tutorial: Input uncertainty quantification. Proc. Winter Simul. Conf. 2014 (IEEE, Piscataway, NJ), 162–176.Google Scholar
  • Song E, Nelson BL, Staum J (2016) Shapley effects for global sensitivity analysis: Theory and computation. SIAM/ASA J. Uncertainty Quantification 4(1):1060–1083.CrossrefGoogle Scholar
  • SPRINT Research Group (2015) A randomized trial of intensive versus standard blood-pressure control. New Engl. J. Med. 373(22):2103–2116.CrossrefGoogle Scholar
  • Stodden V (2015) Reproducing statistical results. Annu. Rev. Stat. Appl. 2(1):1–19.CrossrefGoogle Scholar
  • Tan Z (2006) A distributional approach for causal inference using propensity scores. J. Amer. Statist. Assoc. 101(476):1619–1637.CrossrefGoogle Scholar
  • Taori R, Dave A, Shankar V, Carlini N, Recht B, Schmidt L (2019) When robustness doesn’t promote robustness: Synthetic vs. natural distribution shifts on ImageNet. Preprint submitted September 26, https://openreview.net/forum?id=HyxPIyrFvH.Google Scholar
  • Tipton E, Olsen RB (2018) A review of statistical methods for generalizing from evaluations of educational interventions. Educational Res. 47(8):516–524.CrossrefGoogle Scholar
  • Tipton E, Peck LR (2017) A design-based approach to improve external validity in welfare policy evaluations. Eval. Rev. 41(4):326–356.CrossrefGoogle Scholar
  • Touchette H (2009) The large deviation approach to statistical mechanics. Phys. Rep. 478(1):1–69.CrossrefGoogle Scholar
  • Tsybakov AB (2009) Introduction to Nonparametric Estimation (Springer, New York).CrossrefGoogle Scholar
  • van der Vaart AW, Wellner JA (1996) Weak Convergence and Empirical Processes: With Applications to Statistics (Springer, New York).CrossrefGoogle Scholar
  • Van Mieghem JA (1995) Dynamic scheduling with convex delay costs: The generalized c-μ rule. Ann. Appl. Probab. 5(3):809–833.CrossrefGoogle Scholar
  • Van Parys BP, Esfahani PM, Kuhn D (2021) From data to decisions: Distributionally robust optimization is optimal. Management Sci. 67(6):3387–3402.LinkGoogle Scholar
  • Wainwright MJ (2019) High-Dimensional Statistics: A Non-Asymptotic Viewpoint (Cambridge University Press, Cambridge, UK).CrossrefGoogle Scholar
  • Walton N, Xu K (2021) Learning and information in stochastic networks and queues. Tutorials in Operations Research: Emerging Optimization Methods and Modeling Techniques with Applications (INFORMS, Catonsville, MD), 161–198.LinkGoogle Scholar
  • Wasserman L (2006) All of Nonparametric Statistics (Springer Science & Business Media, New York).Google Scholar
  • Wei D, Ramamurthy KN, Varshney KR (2015) Health insurance market risk assessment: Covariate shift and k−anonymity. SIAM Internat. Conf. Data Mining (SIAM, Philadelphia), 226–234.Google Scholar
  • Weiss A, Shwartz A (1995) Large Deviations for Performance Analysis (Chapman and Hall, Boca Raton, FL).Google Scholar
  • Whitt W (2002) Stochastic Process Limits: An Introduction to Stochastic Process Limits and Their Application to Queues (Springer Science & Business Media, New York).CrossrefGoogle Scholar
  • Williams RJ (2016) Stochastic processing networks. Annu. Rev. Stat. Appl. 3(1):323–345.CrossrefGoogle Scholar
  • Wong A, Otles E, Donnelly JP, Krumm A, McCullough J, DeTroyer-Cooley O, Pestrue J, et al. (2021) External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Intern. Med. 181(8):1065–1070.CrossrefGoogle Scholar
  • Yadlowsky S, Namkoong H, Basu S, Duchi J, Tian L (2022) Bounds on the conditional and average treatment effect with unobserved confounding factors. Ann. Stat. 50(5):2587–2615.CrossrefGoogle Scholar
  • Yu B (1997) Assouad, Fano, and Le Cam. Pollard D, Torgerson E, Yang GL, eds. Festschrift for Lucien Le Cam (Springer, New York), 423–435.CrossrefGoogle Scholar
  • Yu B (2013) Stability. Bernoulli 19(4):1484–1500.CrossrefGoogle Scholar
  • Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, Oermann EK (2018) Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLoS Med. 15(11):e1002683.CrossrefGoogle Scholar
INFORMS site uses cookies to store information on your computer. Some are essential to make our site work; Others help us improve the user experience. By using this site, you consent to the placement of these cookies. Please read our Privacy Statement to learn more.