Robustness of Proactive Intensive Care Unit Transfer Policies

References

• Adelman D, Mersereau AJ (2008) Relaxations of weakly coupled stochastic dynamic programs. Oper. Res. 56(3):712–727.Link, Google Scholar
• Alagoz O, Hsu H, Schaefer AJ, Roberts MS (2010) Markov decision processes: A tool for sequential decision making under uncertainty. Medical Decision Making 30(4):474–483.Crossref, Google Scholar
• Alagoz O, Maillart LM, Schaefer AJ, Roberts MS (2004) The optimal timing of living-donor liver transplantation. Management Sci. 50(10):1420–1430.Link, Google Scholar
• Altman E, Jiménez T, Koole G (2001) On optimal call admission control in resource-sharing system. IEEE Trans. Comm. 49(9):1659–1668.Crossref, Google Scholar
• Ayer T, Alagoz O, Stout NK (2012) OR Forum—a POMDP approach to personalize mammography screening decisions. Oper. Res. 60(5):1019–1034.Link, Google Scholar
• Barnett M, Kaboli P, Sirio CGR (2002) Day of the week of intensive care admission and patient outcomes: A multisite regional evaluation. Medical Care 40(6):530–539.Crossref, Google Scholar
• Barron Y (2016) Performance analysis of a reflected fluid production/inventory model. Math. Methods Oper. Res. 83(1):1–31.Crossref, Google Scholar
• Barron Y (2018) A threshold policy in a Markov-modulated production system with server vacation: The case of continuous and batch supplies. Adv. Appl. Probab. 50(4):1246–1274.Crossref, Google Scholar
• Bilben B, Grandal L, Søvik S (2016) National Early Warning Score (NEWS) as an emergency department predictor of disease severity and 90-day survival in the acutely dyspneic patient—a prospective observational study. Scandinavian J. Trauma Resuscitation Emergency Medicine 24(1):80.Crossref, Google Scholar
• Bountourelis T, Eckman D, Luangkesorn L, Schaefer A, Nabors SG, Clermont G (2012) Sensitivity analysis of an ICU simulation model. Proc. 2012 Winter Simulation Conf. (WSC) (IEEE), 1–12.Google Scholar
• Brunelli A, Ferguson MK, Rocco G, Pieretti P, Vigneswaran WT, Morgan-Hughes NJ, Zanello M, Salati M (2008) A scoring system predicting the risk for intensive care unit admission for complications after major lung resection: A multicenter analysis. Ann. Thoracic Surgery 86(1):213–218.Crossref, Google Scholar
• Caro F, Gupta AD (2022) Robust control of the multi-armed bandit problem. Ann Oper. Res. 317:461–480.Google Scholar
• Chalfin DB, Trzeciak S, Likourezos A, Baumann BM, Dellinger RP; DELAY-ED study group (2007) Impact of delayed transfer of critically ill patients from the emergency department to the intensive care unit. Critical Care Medicine 35(6):1477–1483.Crossref, Google Scholar
• Chan CW, Farias VF, Bambos N, Escobar GJ (2012) Optimizing intensive care unit discharge decisions with patient readmissions. Oper. Res. 60(6):1323–1341.Link, Google Scholar
• Cheng G, Xie J, Zheng Z (2019) Optimal stopping for medical treatment with predictive information. Working paper, National University of Singapore, Singapore.Google Scholar
• Chrusch C, Olafson K, McMillan P, Roberts D, Gray P (2009) High occupancy increases the risk of early death or readmission after transfer from intensive care. Critical Care Medicine 37(10):2753–2758.Google Scholar
• Escobar G, Gardner N, Greene D, Draper D, Kipnis P (2013) Risk-adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Medical Care 51(5):446–453.Crossref, Google Scholar
• Gittins JC (1979) Bandit processes and dynamic allocation indices. J. Roy. Statist. Soc. B 41(2):148–164.Crossref, Google Scholar
• Goh J, Bayati M, Zenios SA, Singh S, Moore D (2018) Data uncertainty in Markov chains: Application to cost-effectiveness analyses of medical innovations. Oper. Res. 66(3):697–715.Link, Google Scholar
• Goyal V, Grand-Clément J (2022) Robust Markov decision processes: Beyond rectangularity. Math. Oper. Res., ePub ahead of print April 20, https://doi.org/10.1287/moor.2022.1259.Google Scholar
• Green L (2002) How many hospitals beds? Inquiry 39(4):400–412.Crossref, Google Scholar
• Heitman SJ, Hilsden RJ, Au F, Dowden S, Manns BJ (2010) Colorectal cancer screening for average-risk North Americans: An economic evaluation. PLoS Medicine 7(11):e1000370.Crossref, Google Scholar
• Higgins TL, Estafanous FG, Loop FD, Lee JC, Starr NJ, Knaus WA, Cosgrove DM III (1997) ICU admission score for predicting morbidity and mortality risk after coronary artery bypass grafting. Ann. Thoracic Surgery 64(4):1050–1058.Crossref, Google Scholar
• Hsu YP (2018) Age of information: Whittle index for scheduling stochastic arrivals. Proc. 2018 IEEE Internat. Sympos. Inform. Theory (ISIT) (IEEE), 2634–2638.Google Scholar
• Hu W, Chan CW, Zubizarreta JR, Escobar GJ (2018) An examination of early transfers to the ICU based on a physiologic risk score. Manufacturing Service Oper. Management 20(3):531–549.Link, Google Scholar
• Iyengar G (2005) Robust dynamic programming. Math. Oper. Res. 30(2):257–280.Link, Google Scholar
• Kallus N, Uehara M (2020) Double reinforcement learning for efficient off-policy evaluation in Markov decision processes. J. Machine Learn. Res. 21(167).Google Scholar
• Kc DS, Terwiesch C (2012) An econometric analysis of patient flows in the cardiac intensive care unit. Manufacturing Service Oper. Management 14(1):50–65.Link, Google Scholar
• Kim MJ, Lim AE (2016) Robust multiarmed bandit problems. Management Sci. 62(1):264–285.Link, Google Scholar
• Kim SH, Chan CW, Olivares M, Escobar G (2014) ICU admission control: An empirical study of capacity allocation and its implication for patient outcomes. Management Sci. 61(1):19–38.Link, Google Scholar
• Kipnis P, Turk BJ, Wulf DA, LaGuardia JC, Liu V, Churpek MM, Romero-Brufau S, Escobar GJ (2016) Development and validation of an electronic medical record-based alert score for detection of inpatient deterioration outside the ICU. J. Biomedical Informatics 64:10–19.Crossref, Google Scholar
• Lowery JC (1992) Simulation of a hospital’s surgical suite and critical care area. Crain RC, ed. Proc. 24th Winter Simulation Conf., Arlington, VA (ACM Press), 1071–1078.Google Scholar
• Madani O, Hanks S, Condon A (1999) On the undecidability of probabilistic planning and infinite-horizon partially observable Markov decision problems. AAAI/IAAI, 541–548.Google Scholar
• Mannor S, Mebel O, Xu H (2016) Robust MDPs with k-rectangular uncertainty. Mathematics Oper. Res. 41(4):1484–1509.Link, Google Scholar
• Mannor S, Simester D, Sun P, Tsitsiklis JN (2007) Bias and variance approximation in value function estimates. Management Sci. 53(2):308–322.Link, Google Scholar
• May WL, Johnson WD (2000) Constructing two-sided simultaneous confidence intervals for multinomial proportions for small counts in a large number of cells. J. Statist. Software 5:1–24.Crossref, Google Scholar
• McClean S, Millard P (2006) Using Markov models to manage high occupancy hospital care. 2006 3rd Internat. IEEE Conf. Intelligent Systems (IEEE), 256–260.Google Scholar
• Milbrandt BAK, Rahim T, Dremsizov T, Clermont G, Cooper M, Angus C, Linde-Zwirble T (2008) Growth of intensive care unit resource use and its estimated cost in medicare. Critical Care Medicine 36(9):2504–2510.Crossref, Google Scholar
• Mullins P, Goyal M, Pines M (2013) National growth in intensive care unit admissions from emergency departments in the United States from 2002 to 2009. Acad. Emergency Medicine 20(5):479–486.Crossref, Google Scholar
• Nilim A, El Ghaoui L (2005) Robust control of Markov decision processes with uncertain transition matrices. Oper. Res. 53(5):780–798.Link, Google Scholar
• Opgenorth D, Stelfox H, Gilfoyle E, Gibney R, Meier M, Boucher P, McKinlay D, Job McIntosh CN, Wang X, Zygun DA, Bagshaw SM (2018) Perspectives on strained intensive care unit capacity: A survey of critical care professionals. PLoS One 13(8):e0201524.Crossref, Google Scholar
• Özekici S, Pliska SR (1991) Optimal scheduling of inspections: A delayed Markov model with false positives and negatives. Oper. Res. 39(2):261–273.Link, Google Scholar
• Peck JS, Benneyan JC, Nightingale DJ, Gaehde SA (2012) Predicting emergency department inpatient admissions to improve same-day patient flow. Acad. Emergency Medicine 19(9):E1045–E1054.Crossref, Google Scholar
• Puterman M (1994) Markov Decision Processes: Discrete Stochastic Dynamic Programming (John Wiley & Sons, New York).Crossref, Google Scholar
• Putnam KG, Buist DS, Fishman P, Andrade SE, Boles M, Chase GA, Goodman MJ, et al. (2002) Chronic disease score as a predictor of hospitalization. Epidemiology 13(3):340–346.Crossref, Google Scholar
• Rapsang AG, Shyam DC (2014) Scoring systems in the intensive care unit: A compendium. Indian J. Critical Care Medicine 18(4):220–228.Crossref, Google Scholar
• Rasouli M, Saghafian S (2018) Robust partially observable Markov decision processes. HKS Working Paper No. RWP18-027, Harvard Kennedy School, Cambridge, MA.Google Scholar
• Shechter SM, Bailey MD, Schaefer AJ, Roberts MS (2008) The optimal time to initiate HIV therapy under ordered health states. Oper. Res. 56(1):20–33.Link, Google Scholar
• Shmueli A, Baras M, Sprung CL (2004) The effect of intensive care on in-hospital survival. Health Services Outcomes Res. Methodology 5(3–4):163–174.Crossref, Google Scholar
• Sison C, Glaz J (1995) Simultaneous confidence intervals and sample size determination for multinomial proportions. J. Amer. Statist. Assoc. 90(429):366–369.Crossref, Google Scholar
• Steimle LN, Denton BT (2017) Markov decision processes for screening and treatment of chronic diseases. Boucherie RJ, van Dijk NM, eds. Markov Decision Processes in Practice (Springer, Berlin), 189–222.Crossref, Google Scholar
• Steimle LN, Kaufman DL, Denton BT (2018) Multi-model Markov decision processes. IISE Trans. 53(10):1124–1139.Google Scholar
• Tripathi V, Modiano E (2019) A Whittle index approach to minimizing functions of age of information. 2019 57th Annual Allerton Conf. Comm. Control Comput. (Allerton) (IEEE), 1160–1167.Google Scholar
• Whittle P (1988) Restless bandits: Activity allocation in a changing world. J. Appl. Probab. 25:287–298.Crossref, Google Scholar
• Wiesemann W, Kuhn D, Rustem B (2013) Robust Markov decision processes. Oper. Res. 38(1):153–183.Abstract, Google Scholar
• Xu H, Mannor S (2012) Distributionally robust Markov decision processes. Math. Oper. Res 37(2):288–300.Link, Google Scholar
• Xu K, Chan CW (2016) Using future information to reduce waiting times in the emergency department via diversion. Manufacturing Service Oper. Management 18(3):314–331.Link, Google Scholar
• Xu Y, Yin W (2013) A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. SIAM J. Imaging Sci. 6(3):1758–1789.Crossref, Google Scholar
• Yauney G, Shah P (2018) Reinforcement learning with action-derived rewards for chemotherapy and clinical trial dosing regimen selection. Proc. Machine Learning Healthcare Conf., 161–226.Google Scholar
• Yu C, Liu J, Nemati S (2021) Reinforcement learning in healthcare: A survey. ACM Comput. Surveys (CSUR) 55(1):1–36.Google Scholar