Smart Testing with Vaccination: A Bandit Algorithm for Active Sampling for Managing COVID-19

References

• Abbasi A, Dillon-Merrill R, Rao HR, Sheng O, Chen R (2021) Call for papers—Special issue of information systems research—Unleashing the power of information technology for strategic management of disasters. Inform. Systems Res. 32(4):1490–1493.Link, Google Scholar
• Ahmad MA, Patel A, Eckert C, Kumar V, Teredesai A (2020) Fairness in machine learning for healthcare. Proc. 26th ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (ACM, New York), 3529–3530.Google Scholar
• Alippi C, Roveri M (2008) Just-in-time adaptive classifiers—Part i: Detecting nonstationary changes. IEEE Trans. Neural Networks 19(7):1145–1153.Crossref, Google Scholar
• Allen H, Tessier E, Turner C, Anderson C, Blomquist P, Simons D, Lochen A, et al. (2022) Comparative transmission of SARS-CoV-2 Omicron (B.1.1.529) and Delta (b.1.617.2) variants and the impact of vaccination: National cohort study, England. Preprint, submitted February 17, https://doi.org/10.1101/2022.02.15.22271001.Google Scholar
• Alon N, Cesa-Bianchi N, Dekel O, Koren T (2015) Online learning with feedback graphs: Beyond bandits. Proc. 28th Conf. Learning Theory, PMLR 40:23–35.Google Scholar
• Asnakew Z, Kerebih Asrese MA (2020) Community risk perception and compliance with preventive measures for COVID-19 pandemic in Ethiopia. Risk Management Healthcare Policy 13:2887–2897.Crossref, Google Scholar
• Atkeson A (2020) What will be the economic impact of COVID-19 in the US? Rough estimates of disease scenarios. NBER Working Paper No. 26867, National Bureau of Economic Research, Cambridge, MA.Google Scholar
• Auer P, Cesa-Bianchi N, Fischer P (2002) Finite-time analysis of the multi-armed bandit problem. Machine Learning 47(2–3):235–256.Crossref, Google Scholar
• Baxter A, Oruc BE, Asplund J, Keskinocak P, Serban N (2022) Evaluating scenarios for school reopening under COVID19. BMC Public Health 22(1):1–10.Google Scholar
• Beca-Martínez MT, Romay-Barja M, Falcón-Romero M, Rodríguez-Blázquez C, Benito-Llanes A, Forjaz MJ (2021) Compliance with the main preventive measures of covid-19 in spain: The role of knowledge, attitudes, practices, and risk perception. Transboundary Emerging Diseases 69(4):e871–e882.Google Scholar
• Besbes O, Gur Y, Zeevi A (2014) Stochastic multi-armed-bandit problem with non-stationary rewards. Adv. Neural Inform. Processing Systems 27.Google Scholar
• Bifet A, Pfahringer B, Read J, Holmes G (2013) Efficient data stream classification via probabilistic adaptive windows. Proc. 28th Annual ACM Sympos. Appl. Comput. (ACM, New York), 801–806.Google Scholar
• Bodas M, Peleg K (2020) Self-isolation compliance in the COVID-19 era influenced by compensation: Findings from a recent survey in Israel: Public attitudes toward the COVID-19 outbreak and self-isolation: A cross sectional study of the adult population of Israel. Health Affairs 39(6):936–941.Crossref, Google Scholar
• Bubeck S, Cesa-Bianchi N (2012) Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations Trends Machine Learn. 5(1):1–122.Google Scholar
• Carpentier A, Valko M (2016) Revealing graph bandits for maximizing local influence. Proc. 19th Internat. Conf. Artificial Intelligence Statistics, PMLR 51:10–18.Google Scholar
• Cervantes A, Gagné C, Isasi P, Parizeau M (2018) Evaluating and characterizing incremental learning from non-stationary data. Preprint, submitted June 18, https://doi.org/10.48550/arXiv.1806.06610.Google Scholar
• Chang SL, Harding N, Zachreson C, Cliff OM, Prokopenko M (2020) Modelling transmission and control of the COVID-19 pandemic in Australia. Nature Comm. 11(1):5710.Google Scholar
• Chapelle O, Li L (2011) An empirical evaluation of Thompson sampling. Adv. Neural Inform. Processing Systems 24:2249–2257.Google Scholar
• Chen H, Huang Z, Li S, Zhang C (2021) Understanding bandits with graph feedback. Adv. Neural Inform. Processing Systems 34:24659–24669.Google Scholar
• Chen X, Li M, Simchi-Levi D, Zhao T (2020) Allocation of COVID-19 vaccines under limited supply. Preprint, submitted August 31, https://dx.doi.org/10.2139/ssrn.3678986.Google Scholar
• Cheung WC, Simchi-Levi D, Zhu R (2022) Hedging the drift: Learning to optimize under non stationarity. Management Sci. 68(3):1696–1713.Link, Google Scholar
• Chiotellis I, Cremers D (2020) Neural online graph exploration. Preprint, submitted December 6, https://doi.org/10.48550/arXiv.2012.03345.Google Scholar
• Chu W, Li L, Reyzin L, Schapire RE (2011) Contextual bandits with linear payoff functions. Proc. 14th Internat.. Conf. Artificial Intelligence Statistics, PMLR 15:208–214.Google Scholar
• Cohn DA, Ghahramani Z, Jordan MI (1996) Active learning with statistical models. J. Artificial Intelligence Res. 4(1):129–145.Crossref, Google Scholar
• Cooper I, Mondal A, Antonopoulos CG (2020) A SIR model assumption for the spread of COVID-19 in different communities. Chaos Solitons Fractals 139:110057.Crossref, Google Scholar
• Dai H, Li Y, Wang C, Singh R, Huang PS, Kohli P (2019) Learning transferable graph exploration. Adv. Neural Inform. Processing Systems 32.Google Scholar
• Dawson P, Werkman M, Brooks-Pollock E, Tildesley M (2015) Epidemic predictions in an imperfect world: Modelling disease spread with partial data. Proc. Royal Soc. B Biol. Sci. 282:20150205.Crossref, Google Scholar
• De Cremer D, Brockner J, Fishman A, Van Dijke M, Van Olffen W, Mayer DM (2010) When do procedural fairness and outcome fairness interact to influence employees’ work attitudes and behaviors? The moderating effect of uncertainty. J. Appl. Psych. 95(2):291.Crossref, Google Scholar
• Eames KT, Keeling MJ (2003) Contact tracing and disease control. Proc. Royal Soc. B Biol. Sci. 270(1533):2565–2571.Crossref, Google Scholar
• Ferretti L, Wymant C, Kendall M, Zhao L, Nurtay A, Abeler-Dörner L, Parker M, Bonsall D, Fraser C (2020) Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing. Science 368(6491):eabb6936.Crossref, Google Scholar
• Franzen A, Wöhner F (2021) Coronavirus risk perception and compliance with social distancing measures in a sample of young adults: Evidence from Switzerland. PLoS One. 16(2):e0247447.Crossref, Google Scholar
• Fylkesnes K, Ndhlovu Z, Kasumba K, Musonda RM, Sichone M (1998) Studying dynamics of the HIV epidemic: Population-based data compared with sentinel surveillance in Zambia. AIDS 12(10):1227–1242.Crossref, Google Scholar
• Garivier A, Cappé O (2011) The KL-UCB algorithm for bounded stochastic bandits and beyond. Proc. 24th Annual Conf. Learning Theory (PMLR, New York) 19:359–376.Google Scholar
• Garivier A, Moulines E (2011) On upper-confidence bound policies for switching bandit problems. Internat. Conf. Algorithmic Learning Theory (Springer, Berlin), 174–188.Google Scholar
• Goel S, Salganik MJ (2010) Assessing respondent-driven sampling. Proc. Natl. Acad. Sci. USA 107(15):6743–6747.Crossref, Google Scholar
• Grushka-Cohen H, Cohen R, Shapira B, Moran-Gilad J, Rokach L (2020) A framework for optimizing covid-19 testing policy using a multi armed bandit approach. Preprint, submitted July 28, https://doi.org/10.48550/arXiv.2007.14805.Google Scholar
• Gu Q, Han J (2014) Online spectral learning on a graph with bandit feedback. Proc. 2014 IEEE Internat. Conf. Data Mining (IEEE, Piscataway, NJ), 833–838.Google Scholar
• Hellewell J, Abbott S, Gimma A, Bosse NI, Jarvis CI, Russell TW, Munday JD, et al. (2020) Feasibility of controlling covid-19 outbreaks by isolation of cases and contacts. Lancet Global Health 8(4):E488–E496.Crossref, Google Scholar
• Henderson K, Gallagher B, Eliassi-Rad T, Tong H, Basu S, Akoglu L, Koutra D, Faloutsos C, Li L (2012) RolX: Structural role extraction & mining in large graphs. Proc. 18th ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (ACM, New York), 1231–1239.Google Scholar
• Hethcote HW (2000) The mathematics of infectious diseases. SIAM Rev. 42(4):599–653.Crossref, Google Scholar
• Hoertel N, Blachier M, Blanco C, Olfson M, Massetti M, Limosin F, Leleu H (2020) Facing the COVID-19 epidemic in NYC: A stochastic agent-based model of various intervention strategies. Preprint, submitted April 28, https://doi.org/10.1101/2020.04.23.20076885.Google Scholar
• Huerta R, Tsimring LS (2002) Contact tracing and epidemics control in social networks. Physical Rev. E 66:056115.Crossref, Google Scholar
• Jecker NS (2021) Cash incentives, ethics, and COVID-19 vaccination. Science 374(6569):819–820.Crossref, Google Scholar
• Jiang H (2019) Non-asymptotic uniform rates of consistency for k-NN regression. Proc. AAAI Conf. Artificial Intelligence (AAAI Press, Palo Alto, CA), 3999–4006.Google Scholar
• Kamarthi H, Vijayan P, Wilder B, Ravindran B, Tambe M (2020) Influence maximization in unknown social networks: Learning policies for effective graph sampling. Proc. 19th Internat. Conf. Autonomous Agents and MultiAgent Systems, 575–583.Google Scholar
• Kandasamy K, Krishnamurthy A, Schneider J, Póczos B (2018) Parallelised Bayesian optimisation via Thompson sampling. Proc. 21st Internat. Conf. Artificial Intelligence Statistics (PMLR, New York), 133–142.Google Scholar
• Koo JR, Cook AR, Park M, Sun Y, Sun H, Lim JT, Tam C, Dickens BL (2020) Interventions to mitigate early spread of SAR-CoV-2 in Singapore: A modelling study. Lancet Infectious Diseases 20(6):678–688.Crossref, Google Scholar
• Lattimore T, Szepesvári C (2020) Bandit Algorithms (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• Lauer SA, Grantz KH, Bi Q, Jones FK, Zheng Q, Meredith HR, Azman AS, Reich NG, Lessler J (2020) The incubation period of coronavirus disease 2019 (COVID-19) from publicly reported confirmed cases: Estimation and application. Ann. Internal Medicine 172(9):577–582.Crossref, Google Scholar
• Lee LM, Thacker SB, Louis MES (2010) Principles and Practice of Public Health Surveillance (Oxford University Press, New York).Crossref, Google Scholar
• Li L, Chu W, Langford J, Schapire RE (2010) A contextual-bandit approach to personalized news article recommendation. Proc. 19th Internat. Conf. World Wide Web (ACM, New York), 661–670.Google Scholar
• Long EF, Nohdurft E, Spinler S (2018) Spatial resource allocation for emerging epidemics: A comparison of greedy, myopic, and dynamic policies. Manufacturing Service Oper. Management 20(2):181–198.Link, Google Scholar
• Ma Y, Huang TK, Schneider JG (2015) Active search and bandits on graphs using sigma-optimality. Proc. 31st Conf. Uncertainty Artificial Intelligence (AUAI, Amsterdam), 542–551.Google Scholar
• Macready WG, Wolpert DH (1998) Bandit problems and the exploration/exploitation tradeoff. IEEE Trans. Evolutionary Comput. 2(1):2–22.Crossref, Google Scholar
• Madewell ZJ, Yang Y, Longini IM, Halloran ME, Dean NE (2020) Household transmission of SARS-CoV-2: A systematic review and meta-analysis. JAMA Network Open 3(12):e2031756.Crossref, Google Scholar
• Madhawa K, Murata T (2019) A multi-armed bandit approach for exploring partially observed networks. Appl. Network Sci. 4(1):26.Crossref, Google Scholar
• Magnani R, Sabin K, Saidel T, Heckathorn D (2005) Review of sampling hard-to-reach and hidden populations for HIV surveillance. AIDS 19:S67–S72.Crossref, Google Scholar
• Mallapaty S (2022) COVID-19: How Omicron overtook Delta in three charts. Nature 10.Google Scholar
• Manchein C, Brugnago EL, da Silva RM, Mendes CF, Beims MW (2020) Strong correlations between power-law growth of covid-19 in four continents and the inefficiency of soft quarantine strategies. Chaos 30(4):041102.Crossref, Google Scholar
• Mannor S, Shamir O (2011) From bandits to experts: On the value of side-observations. Adv. Neural Inform. Processing Systems 24:684–692.Google Scholar
• Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. Adv. Neural Inform. Processing Systems 26:3111–3119.Google Scholar
• Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, et al. (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533.Crossref, Google Scholar
• Murray CJ (2022) COVID-19 will continue but the end of the pandemic is near. Lancet 399(10323):417–419.Crossref, Google Scholar
• Nemhauser GL, Wolsey LA, Fisher ML (1978) An analysis of approximations for maximizing submodular set functions—I. Math. Programming 14(1):265–294.Crossref, Google Scholar
• Ning L, Niu J, Bi X, Yang C, Liu Z, Wu Q, Ning N, et al. (2020) The impacts of knowledge, risk perception, emotion and information on citizens’ protective behaviors during the outbreak of COVID-19: A cross-sectional study in China. BMC Public Health 20(1):1–12.Crossref, Google Scholar
• Perez L, Dragicevic S (2009) An agent-based approach for modeling dynamics of contagious disease spread. Internat. J. Health Geographics 8(1):50.Crossref, Google Scholar
• Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: Online learning of social representations. Proc. 20th ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (ACM, New York), 701–710.Google Scholar
• Prem K, Liu Y, Russell TW, Kucharski AJ, Eggo RM, Davies N, Flasche S, et al. (2020) The effect of control strategies to reduce social mixing on outcomes of the COVID-19 epidemic in Wuhan, China: A modelling study. Lancet Public Health 5(5):e261–e270.Crossref, Google Scholar
• Provodin D, Gajane P, Pechenizkiy M, Kaptein M (2022) The impact of batch learning in stochastic linear bandits. Preprint, submitted February 14, https://doi.org/10.48550/arXiv.2202.06657.Google Scholar
• Raj V, Kalyani S (2017) Taming non-stationary bandits: A Bayesian approach. Preprint, submitted July 31, https://doi.org/10.48550/arXiv.1707.09727.Google Scholar
• Roda WC, Varughese MB, Han D, Li MY (2020) Why is it difficult to accurately predict the COVID-19 epidemic? Infectious Disease Modelling 5:271–281.Crossref, Google Scholar
• Saar-Tsechansky M, Provost FJ (2007) Decision-centric active learning of binary-outcome models. Inform. Systems Res. 18(1):4–22.Link, Google Scholar
• Salathé M, Althaus CL, Neher R, Stringhini S, Hodcroft E, Fellay J, Zwahlen M, et al. (2020) COVID-19 epidemic in Switzerland: On the importance of testing, contact tracing and isolation. Swiss Medical Weekly 150(11–12):w20225.Google Scholar
• Silva PC, Batista PV, Lima HS, Alves MA, Guimarães FG, Silva RC (2020) COVID-ABS: An agent-based model of COVID-19 epidemic to simulate health and economic effects of social distancing interventions. Chaos Solitons Fractals 139:110088.Crossref, Google Scholar
• Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, et al. (2016) Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484–489.Crossref, Google Scholar
• Singh R, Liu F, Liu X, Shroff N (2020) Contextual bandits with side-observations. Preprint, submitted June 6, https://doi.org/10.48550/arXiv.2006.03951.Google Scholar
• Singla A, Horvitz E, Kohli P, White R, Krause A (2015) Information gathering in networks via active exploration. Proc. 24th Internat. Conf. Artificial Intelligence (AAAI Press, Palo Alto, CA), 981–988.Google Scholar
• Soundarajan S, Eliassi-Rad T, Gallagher B, Pinar A (2017) ε-WGX: Adaptive edge probing for enhancing incomplete networks. Proc. 2017 ACM on Web Science Conf. (ACM, New York), 161–170.Google Scholar
• Sutton RS, Barto AG (2018) Reinforcement Learning: An Introduction (MIT Press, Cambridge, MA).Google Scholar
• Thompson WR (1933) On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25(3–4):285–294.Crossref, Google Scholar
• Topirceanu A, Udrescu M, Marculescu R (2020) Centralized and decentralized isolation strategies and their impact on the COVID-19 pandemic dynamics. Preprint, submitted April 8, https://doi.org/10.48550/arXiv.2004.04222.Google Scholar
• Trovo F, Paladino S, Restelli M, Gatti N (2020) Sliding-window Thompson sampling for non-stationary settings. J. Artificial Intelligence Res. 68:311–364.Crossref, Google Scholar
• Ulloa AC, Buchan SA, Daneman N, Brown KA (2022) Estimates of SARS-CoV-2 Omicron variant severity in Ontario, Canada. JAMA 327(13):1286–1288.Crossref, Google Scholar
• Underwood BR, White VLC, Baker T, Law M, Moore-Gillon JC (2003) Contact tracing and population screening for tuberculosis – Who should be assessed? J. Public Health 25(1):59–61.Crossref, Google Scholar
• Wang Z, Schaul T, Hessel M, Hasselt H, Lanctot M, Freitas N (2016) Dueling network architectures for deep reinforcement learning. Proc. 33rd Internat. Conf. Machine Learning, PMLR 48:1995–2003.Google Scholar
• Whitney R, Viswanath K (2004) Lessons learned from public health mass media campaigns: Marketing health in a crowded media world. Annual Rev. Public Health 25:419.Crossref, Google Scholar
• World Meter (2020) COVID-19 coronavirus pandemic. Retrieved August 31 from https://www.worldometers.info/coronavirus/.Google Scholar
• Zheng Z, Padmanabhan B (2006) Selectively acquiring customer information: A new data acquisition problem and an active learning-based solution. Management Sci. 52(5):697–712.Link, Google Scholar