Spoiled for Choice? Personalized Recommendation for Healthcare Decisions: A Multiarmed Bandit Approach

References

• Abbasi A, Sarker S, Chiang RH (2016) Big data research in information systems: Toward an inclusive research agenda. J. Assoc. Inform. Systems 17(2):3.Google Scholar
• Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Trans. Knowledge Data Engrg. 17(6):734–749.Crossref, Google Scholar
• Adomavicius G, Tuzhilin A (2011) Context-aware recommender systems. Ricci F, Rokach L, Shapira B, Kantor PB, eds. Recommender Systems Handbook (Springer, Berlin), 217–253.Crossref, Google Scholar
• An M, Wu F, Wu C, Zhang K, Liu Z, Xie X (2019) Neural news recommendation with long-and short-term user representations. Korhonen A, Traum D, Màrquez L, eds. Proc. 57th Annual Meeting. Assoc. Comput. Linguistics (Association for Computational Linguistics, Florence), 336–345.Google Scholar
• Auer P, Cesa-Bianchi N, Fischer P (2002) Finite-time analysis of the multiarmed bandit problem. Machine Learn. 47(2–3):235–256.Crossref, Google Scholar
• Baltrunas L, Ludwig B, Ricci F (2011) Matrix factorization techniques for context aware recommendation. Proc. Fifth ACM Conf. Recommender Systems (ACM, New York), 301–304.Google Scholar
• Bandura A (1991) Social cognitive theory of self-regulation. Organ. Behav. Human Decision Processes 50(2):248–287.Crossref, Google Scholar
• Bandura A (2004) Health promotion by social cognitive means. Health Ed. Behav. 31(2):143–164.Crossref, Google Scholar
• Barello S, Triberti S, Graffigna G, Libreri C, Serino S, Hibbard J, Riva G (2016) eHealth for patient engagement: A systematic review. Frontiers. Psych. 6:2013. https://psycnet.apa.org/record/2016-18429-001.Google Scholar
• Bateman PJ, Gray PH, Butler BS (2011) Research note—The impact of community commitment on participation in online communities. Inform. Systems Res. 22(4):841–854.Link, Google Scholar
• California HealthCare Foundation (2017) Consumers in healthcare: The burden of choice. Accessed October 30, 2019, https://www.chcf.org/wp-content/uploads/2017/12/PDF-ConsumersInHealthCareBurdenChoice.pdf.Google Scholar
• Centers for Disease Control and Prevention (2019) About chronic diseases. Accessed October 23, 2019, https://www.cdc.gov/chronicdisease/about/index.htm#:∼:text=Many%20chronic%20diseases%20are%20caused,Lack%20of%20physical%20activity.Google Scholar
• Chapelle O, Li L (2011) An empirical evaluation of Thompson sampling. Adv. Neural Inform. Processing Systems 24:2249–2257.Google Scholar
• Chen A (2005) Context-aware collaborative filtering system: Predicting the user’s preference in the ubiquitous computing environment. Internat. Sympo. Location Context Awareness (Springer), 244–253.Google Scholar
• Chen H, Chiang RH, Storey VC (2012) Business intelligence and analytics: From big data to big impact. Management Inform. Systems Quart. 36(4):1165–1188.Crossref, Google Scholar
• Cheng H-T, Koc L, Harmsen J, Shaked T, Chandra T, Aradhye H, Anderson G, Corrado G, Chai W, Ispir M (2016) Wide & deep learning for recommender systems. Proc. First Workshop Deep Learn. Recommender Systems, 7–10.Google Scholar
• Coelho JJ, Arnold A, Nayler J, Tischkowitz M, MacKay J (2005) An assessment of the efficacy of cancer genetic counselling using real-time videoconferencing technology (telemedicine) compared with face-to-face consultations. Eur. J. Cancer Care (England) 41(15):2257–2261.Crossref, Google Scholar
• Cremonesi P, Koren Y, Turrin R (2010) Performance of recommender algorithms on top-n recommendation tasks. Proc. Fourth ACM Conf. Recommender Systems, 39–46.Google Scholar
• Cutler DM (2004) Behavioral health interventions: What works and why. Critical Perspectives on Racial and Ethnic Differences in Health in Late Life, 643–674.Google Scholar
• DiMatteo MR (2004) Social support and patient adherence to medical treatment: A meta-analysis. Health Psych. 23(2):207–218.Crossref, Google Scholar
• Doran GT (1981) There’s a SMART way to write management’s goals and objectives. Management Rev. 70(11):35–36.Google Scholar
• Dudík M, Langford J, Li L (2011) Doubly robust policy evaluation and learning. Preprint, submitted March 23, https://arxiv.org/abs/1103.4601.Google Scholar
• Eysenbach G (2005) The law of attrition. J. Medical Internet Res. 7(1).Crossref, Google Scholar
• Fuglede B, Topsoe F (2004) Jensen-Shannon divergence and Hilbert space embedding. Proc. Internat. Sympos. Inform. Theory (IEEE), 31.Google Scholar
• Gittins JC (1979) Bandit processes and dynamic allocation indices. J. Roy. Statist. Soc. B 41(2):148–164.Crossref, Google Scholar
• Hibbard JH, Mahoney ER, Stock R, Tusler M (2007) Do increases in patient activation result in improved self‐management behaviors? Health Services Res. 42(4):1443–1463.Crossref, Google Scholar
• Hidasi B, Karatzoglou A, Baltrunas L, Tikk D (2016) Session-Based Recommendations with Recurrent Neural Networks (ICLR, San Juan, Puerto Rico).Google Scholar
• Houlihan S (2018) Dual-process models of health-related behaviour and cognition: A review of theory. Public Health 156:52–59.Crossref, Google Scholar
• Howard J, Gugger S (2020) Fastai: A layered API for deep learning. Inform. (Basel) 11(2):108.Google Scholar
• Jiang N, Krishnamurthy A, Agarwal A, Langford J, Schapire RE (2017) Contextual decision processes with low Bellman rank are PAC-learnable. Internat. Conf. Machine Learn. (PMLR), 1704–1713.Google Scholar
• Johnson F, Wardle J (2011) The association between weight loss and engagement with a web-based food and exercise diary in a commercial weight loss programme: A retrospective analysis. Internat. J. Behav. Nutrition Physical Activity 8(1):83.Crossref, Google Scholar
• Johnson PE, Veazie PJ, Kochevar L, O’Connor PJ, Potthoff SJ, Verma D, Dutta P (2002) Understanding variation in chronic disease outcomes. Health Care Management Sci. 5(3):175–189.Crossref, Google Scholar
• King G, Willoughby C, Specht JA, Brown E (2006) Social support processes and the adaptation of individuals with chronic disabilities. Qualitative Health Res. 16(7):902–925.Crossref, Google Scholar
• Kovacs G, Wu Z, Bernstein MS (2018) Rotating online behavior change interventions increases effectiveness but also increases attrition. Proc. ACM Human Comput. Interaction (CSCW), 1–25.Google Scholar
• Krukowski RA, Harvey-Berino J, Ashikaga T, Thomas CS, Micco N (2008) Internet-based weight control: The relationship between web features and weight loss. Telemedicine J. E-Health 14(8):775–782.Crossref, Google Scholar
• Lei H, Tewari A, Murphy SA (2017) An actor-critic contextual bandit algorithm for personalized mobile health interventions. Preprint, submitted June 28, https://arxiv.org/abs/1706.09090.Google Scholar
• Locke EA, Latham GP (1990) A Theory of Goal Setting & Task Performance (Prentice-Hall, Inc., Hoboken, NJ).Google Scholar
• Lorig K, Ritter PL, Laurent DD, Plant K, Green M, Jernigan VBB, Case S (2010) Online diabetes self-management program: A randomized study. Diabetes Care 33(6):1275–1281.Crossref, Google Scholar
• Lupton D (2013) The digitally engaged patient: Self-monitoring and self-care in the digital health era. Soc. Theory Health 11(3):256–270.Crossref, Google Scholar
• McLean V (2011) Motivating patients to use smartphone health apps. PR Web 113. Accessed November 24, 2019, http://www.prweb.com/releases/2011/04/prweb5268884.htm.Google Scholar
• Mnih A, Salakhutdinov RR (2008) Probabilistic matrix factorization. Adv. Neural Inform. Processing Systems 20:1257–1264.Google Scholar
• Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533.Crossref, Google Scholar
• Morid MA, Sheng ORL, Dunbar J (2022) Time series prediction using deep learning methods in healthcare. ACM Trans. Management Inform. Systems, ePub ahead of print April 22, https://doi.org/10.1145/3531326.Google Scholar
• Nahum-Shani I, Smith SN, Spring BJ, Collins LM, Witkiewitz K, Tewari A, Murphy SA (2018) Just-in-time adaptive interventions (JITAIs) in mobile health: Key components and design principles for ongoing health behavior support. Ann. Behav. Medicine 52(6):446–462.Crossref, Google Scholar
• Nguyen K, Daumé H III, Boyd-Graber J (2017) Reinforcement learning for bandit neural machine translation with simulated human feedback. Preprint, submitted July 24, https://arxiv.org/abs/1707.07402.Google Scholar
• Ogbeiwi O (2018) General concepts of goals and goal-setting in healthcare: A narrative review. J. Management Organ. 27(2):324–341.Crossref, Google Scholar
• Oulasvirta A, Hukkinen JP, Schwartz B (2009) When more is less: The paradox of choice in search engine use. Proc. 32nd Internat. ACM SIGIR Conf. Res. Development Inform. Retrieval, 516–523.Google Scholar
• Paredes P, Gilad-Bachrach R, Czerwinski M, Roseway A, Rowan K, Hernandez J (2014) PopTherapy: Coping with stress through pop-culture. Proc. Eighth Internat. Conf. Pervasive Comput. Tech. Healthcare, 109–117.Google Scholar
• Rabbi M, Pfammatter A, Zhang M, Spring B, Choudhury T (2015) Automated personalized feedback for physical activity and dietary behavior change with mobile phones: A randomized controlled trial on adults. JMIR Mhealth Uhealth 3(2):e4160.Crossref, Google Scholar
• Sedhain S, Sanner S, Braziunas D, Xie L, Christensen J (2014) Social collaborative filtering for cold-start recommendations. Proc. Eighth ACM Conf. Recommender Systems, 345–348.Google Scholar
• Short SE, Mollborn S (2015) Social determinants and health behaviors: Conceptual frames and empirical advances. Current Opinion Psych. 5:78–84.Crossref, Google Scholar
• Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529(7587):484–489.Crossref, Google Scholar
• Sutton RS, Barto AG (2018) Reinforcement Learning: An Introduction (MIT Press, Cambridge, MA).Google Scholar
• Tang J, Wang K (2018) Personalized top-n sequential recommendation via convolutional sequence embedding. Proc. 11th ACM Internat. Conf. Web Search Data Mining, 565–573.Google Scholar
• Tate DF, Finkelstein EA, Khavjou O, Gustafson A (2009) Cost effectiveness of internet interventions: Review and recommendations. Ann. Behav. Medicine 38(1):40–45.Crossref, Google Scholar
• Tewari A, Murphy SA (2017) From ads to interventions: Contextual bandits in mobile health. Mobile Health (Springer), 495–517.Crossref, Google Scholar
• Tomkins S, Liao P, Klasnja P, Murphy S (2021) IntelligentPooling: Practical Thompson sampling for mHealth. Machine Learn. 110:2685–2727.Crossref, Google Scholar
• Valcarce D, Bellogín A, Parapar J, Castells P (2018) On the robustness and discriminative power of information retrieval metrics for top-N recommendation. Proc. 12th ACM Conf. Recommender Systems (ACM, New York), 260–268.Google Scholar
• van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J. Machine Learn. Res. 9:2579–2605.Google Scholar
• West R, Raw M, McNeill A, Stead L, Aveyard P, Bitton J, Stapleton J, McRobbie H, Pokhrel S, Lester‐George A (2015) Health‐care interventions to promote and assist tobacco cessation: A review of efficacy, effectiveness and affordability for use in national guideline development. Addiction 110(9):1388–1403.Crossref, Google Scholar
• White A, Kavanagh D, Stallman H, Klein B, Kay-Lambkin F, Proudfoot J, Drennan J, Connor J, Baker A, Hines E (2010) Online alcohol interventions: A systematic review. J. Medical Internet Res. 12(5):e1479.Crossref, Google Scholar
• Wu C, Wu F, An M, Huang J, Huang Y, Xie X (2019) NPA: Neural news recommendation with personalized attention. Proc. 25th ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining, 2576–2584.Google Scholar
• Yan L (2018) Good intentions, bad outcomes: The effects of mismatches between social support and health outcomes in an online weight loss community. Production Oper. Management 27(1):9–27.Crossref, Google Scholar
• Yan L, Tan Y (2014) Feeling blue? Go online: An empirical study of social support among patients. Inform. Systems Res. 25(4):690–709.Link, Google Scholar
• Ybarra ML, Eaton WW (2005) Internet-based mental health interventions. Mental Health Services Res. 7(2):75–87.Crossref, Google Scholar
• Yu Z, Lian J, Mahmoody A, Liu G, Xie X (2019) Adaptive user modeling with long and short-term preferences for personalized recommendation. Proc. 28th Internat. Joint Conf. Artificial Intelligence, 4213–4219.Google Scholar
• Yuan F, Karatzoglou A, Arapakis I, Jose JM, He X (2019) A simple convolutional generative network for next item recommendation. Proc. 12th ACM Internat. Conf. Web Search Data Mining, 582–590.Google Scholar
• Zaman N, Li J (2014) Semantics-enhanced recommendation system for social healthcare. 2014 IEEE 28th Internat. Conf. Adv. Inform. Networking Appl. (IEEE, Piscataway, NJ), 765–770.Google Scholar
• Zhang S, Yao L, Sun A, Tay Y (2019) Deep learning based recommender system: A survey and new perspectives. ACM Comput. Surveys 52(1):1–38.Crossref, Google Scholar
• Zhao X, Xia L, Tang J, Yin D (2019) Deep reinforcement learning for search, recommendation, and online advertising: A survey. ACM Sigweb Newsletter, 1–15.Google Scholar
• Zhou T, Yan L, Wang Y, Tan Y (2021) Turn your online weight management from zero to hero: A multidimensional, continuous-time evaluation. Management Sci. 68(5):3507–3527.Link, Google Scholar