Multi-Armed Bandits with Endogenous Learning Curves: An Application to Split Liver Transplantation

References

• Akan M, Alagoz O, Ata B, Erenay FS, Said A (2012) A broader view of designing the liver allocation system. Oper. Res. 60(4):757–770.Link, Google Scholar
• Alban A, Chick SE, Zoumpoulis SI (2022) Learning personalized treatment strategies with predictive and prognostic covariates in adaptive clinical trials. Preprint, submitted July 21, http://dx.doi.org/10.2139/ssrn.4160045.Google Scholar
• Anderer A, Bastani H, Silberholz J (2022) Adaptive clinical trial designs with surrogates: When should we bother? Management Sci. 68(3):1982–2002.Link, Google Scholar
• Arlotto A, Chick SE, Gans N (2014) Optimal hiring and retention policies for heterogeneous workers who learn. Management Sci. 60(1):110–129.Link, Google Scholar
• Ban GY, Keskin NB (2021) Personalized dynamic pricing with machine learning: High-dimensional features and heterogeneous elasticity. Management Sci. 67(9):5549–5568.Link, Google Scholar
• Bertsimas D, Farias VF, Trichakis N (2011) The price of fairness. Oper. Res. 59(1):17–31.Link, Google Scholar
• Bertsimas D, Papalexopoulos T, Trichakis N, Wang Y, Hirose R, Vagefi PA (2020) Balancing efficiency and fairness in liver transplant access: Tradeoff curves for the assessment of organ distribution policies. Transplantation 104(5):981–987.Crossref, Google Scholar
• Besbes O, Gur Y, Zeevi A (2019) Optimal exploration–Exploitation in a multi-armed bandit problem with non-stationary rewards. Stochastic Systems 9(4):319–337.Link, Google Scholar
• Cheung WC, Simchi-Levi D, Zhu R (2020) Reinforcement learning for non-stationary Markov decision processes: The blessing of (more) optimism. Daumé H III, Aarti S, eds. Proc. 37th Internat. Conf. Machine Learn., Proceedings of Machine Learning Research, vol. 119 (PMLR, New York), 1843–1854.Google Scholar
• Chick SE, Gans N, Yapar Ö (2022) Bayesian sequential learning for clinical trials of multiple correlated medical interventions. Management Sci. 68(7):4919–4938.Link, Google Scholar
• den Boer AV, Keskin NB (2022) Dynamic pricing with demand learning and reference effects. Management Sci. 68(10):7112–7130.Link, Google Scholar
• Duke H (2021) Duke health blog. Accessed March 20, 2023, https://www.dukehealth.org/blog/split-liver-transplant-saves-two-lives-one-donor-liver.Google Scholar
• Emre S, Umman V (2011) Split liver transplantation: An overview. Transplantation Proc. 43(3):884–887.Crossref, Google Scholar
• Garivier A, Moulines E (2011) On upper-confidence bound policies for switching bandit problems. Kivinen J, Szepesvári C, Ukkonen E, Zeugmann T, eds. Algorithmic Learning Theory. ALT 2011, Lecture Notes in Computer Science, vol. 6925 (Springer, Berlin, Heidelberg), 174–188.Google Scholar
• Garivier A, Ménard P, Stoltz G (2019) Explore first, exploit next: The true shape of regret in bandit problems. Math. Oper. Res. 44(2):377–399.Link, Google Scholar
• Ge J, Perito ER, Bucuvalas J, Gilroy R, Hsu EK, Roberts JP, Lai JC (2020) Split liver transplantation is utilized infrequently and concentrated at few transplant centers in the United States. Amer. J. Transplantation 20(4):1116–1124.Crossref, Google Scholar
• Goodfellow I, Bengio Y, Courville A (2016) Deep Learning (MIT Press, Cambridge, MA).Google Scholar
• Grover A, Markov T, Attia P, Jin N, Perkins N, Cheong B, Chen M, et al. (2018) Best arm identification in multi-armed bandits with delayed feedback. Storkey A, Perez-Cruz F, eds. Proc. Twenty-First Internat. Conf. Artificial Intelligence Statistics, Proceedings of Machine Learning Research, vol. 84 (PMLR, New York), 833–842.Google Scholar
• Hackl C, Schmidt KM, Süsal C, Döhler B, Zidek M, Schlitt HJ (2018) Split liver transplantation: Current developments. World J. Gastroenterology 24(47):5312–5321.Crossref, Google Scholar
• Hansen N (2006) The CMA evolution strategy: A comparing review. Lozano JA, Larrañaga P, Inza I, Bengoetxea E, eds. Towards a New Evolutionary Computation, Studies in Fuzziness and Soft Computing, vol. 192 (Springer, Berlin, Heidelberg), 75–102.Crossref, Google Scholar
• Hardy GH (1952) Inequalities (Cambridge University Press, Cambridge, UK).Google Scholar
• Joulani P, Gyorgy A, Szepesvári C (2013) Online learning under delayed feedback. ICML’13: Proc. 30th Internat. Conf. Machine Learning (JMLR), 1453–1461.Google Scholar
• Kantidakis G, Putter H, Lancia C, Boer J, Braat AE, Fiocco M (2020) Survival prediction models since liver transplantation-Comparisons between Cox models and machine learning techniques. BMC Medical Res. Methodology 20(1):277.Crossref, Google Scholar
• Keskin NB, Li M (2021) Selling quality-differentiated products in a Markovian market with unknown transition probabilities. Oper. Res. 72(3):885–902.Google Scholar
• Keskin NB, Zeevi A (2017) Chasing demand: Learning and earning in a changing environment. Math. Oper. Res. 42(2):277–307.Link, Google Scholar
• Keskin NB, Li Y, Sunar N (2024) Data-driven clustering and feature-based retail electricity pricing with smart meters. Oper. Res., ePub ahead of print September 3, https://doi.org/10.1287/opre.2022.0112.Link, Google Scholar
• Lattimore T, Szepesvári C (2020) Bandit Algorithms (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• Le Morvan P, Stock B (2005) Medical learning curves and the Kantian ideal. J. Medical Ethics 31(9):513–518.Crossref, Google Scholar
• Lehmann EL, Casella G (2006) Theory of Point Estimation (Springer-Verlag, New York).Google Scholar
• McDiarmid C (1998) Concentration. Habib M, McDiarmid C, Ramirez-Alfonsin J, Reed B, eds. Probabilistic Methods for Algorithmic Discrete Mathematics, Algorithms and Combinatorics, vol. 16 (Springer, Berlin, Heidelberg), 195–248.Crossref, Google Scholar
• Nitski O, Azhie A, Qazi-Arisar FA, Wang X, Ma S, Lilly L, Watt KD, et al. (2021) Long-term mortality risk stratification of liver transplant recipients: Real-time application of deep learning algorithms on longitudinal data. Lancet Digital Health 3(5):e295–e305.Crossref, Google Scholar
• OPTN/UNOS Ethics Committee (2016) Split versus whole liver transplantation. Accessed January 25, 2025, https://optn.transplant.hrsa.gov/professionals/by-topic/ethical-considerations/split-versus-whole-liver-transplantation/.Google Scholar
• Perito ER, Roll G, Dodge JL, Rhee S, Roberts JP (2019) Split liver transplantation and pediatric waitlist mortality in the United States: Potential for improvement. Transplantation 103(3):552–557.Crossref, Google Scholar
• Pusic MV, Boutis K, Hatala R, Cook DA (2015) Learning curves in health professions education. Acad. Medicine 90(8):1034–1042.Crossref, Google Scholar
• Rawls J (2001) Justice as Fairness: A Restatement (Harvard University Press, Cambridge, MA).Crossref, Google Scholar
• Schumann C, Lang Z, Mattei N, Dickerson JP (2019) Group fairness in bandit arm selection. Preprint, submitted December 9, https://arxiv.org/abs/1912.03802.Google Scholar
• Thompson WR (1933) On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25(3/4):285–294.Crossref, Google Scholar
• UNOS (2020) United networks of organ sharing data. Accessed August 3, 2024, https://optn.transplant.hrsa.gov/data/view-data-reports/national-data/.Google Scholar
• Vulchev A, Roberts JP, Stock PG (2004) Ethical issues in split versus whole liver transplantation. Amer. J. Transplantation 4(11):1737–1740.Crossref, Google Scholar
• Zenios SA, Chertow GM, Wein LM (2000) Dynamic allocation of kidneys to candidates on the transplant waiting list. Oper. Res. 48(4):549–569.Link, Google Scholar