Online Modeling and Monitoring for Dependent Dynamic Processes Under Resource Constraints

References

• Abbasi-Yadkori Y, Pál D, Szepesvári C (2011) Improved algorithms for linear stochastic bandits. Shawe-Taylor J, Zemel R, Bartlett P, Pereira F, Weinberger K, eds. Adv. Neural Inform. Processing Systems, vol. 24 (Curran Associates, Inc., Red Hook, NY), 2312–2320.Google Scholar
• Abbou A, Makis V (2019) Group maintenance: A restless bandits approach. INFORMS J. Comput. 31(4):719–731.Link, Google Scholar
• Ahuja V, Birge JR (2020) An approximation approach for response-adaptive clinical trial design. Informs J. Comput. 32(4):877–894.Abstract, Google Scholar
• Alaa AM, Van Der Schaar M (2018) A hidden absorbing semi-Markov model for informatively censored temporal data: Learning and inference. J. Machine Learn. Res. 19(4):1–62.Google Scholar
• Ancira-Moreno M, Vadillo-Ortega F, Rivera-Dommarco JÁ, Sánchez BN, Pasteris J, Batis C, Castillo-Castrejón M, O’Neill MS (2019) Gestational weight gain trajectories over pregnancy and their association with maternal diet quality: Results from the PRINCESA cohort. Nutrition 65:158–166.Google Scholar
• Arevalo-Rodriguez I, Smailagic N, I Figuls MR, Ciapponi A, Sanchez-Perez E, Giannakou A, Pedraza OL, Cosp XB, Cullum S (2015) Mini-mental state examination (MMSE) for the detection of Alzheimer’s disease and other dementias in people with mild cognitive impairment (MCI). Cochrane Database Systematic Rev. 3:CD010783.Google Scholar
• AshrafVirk MuR, Mysorewala MF, Cheded L, Aliyu A. (2022) Review of energy harvesting techniques in wireless sensor-based pipeline monitoring networks. Renewable Sustainable Energy Rev. 157:112046.Google Scholar
• Auer P (2002) Using confidence bounds for exploitation-exploration trade-offs. J. Machine Learn. Res. 3:397–422.Google Scholar
• Auer P, Cesa-Bianchi N, Fischer P (2002) Finite-time analysis of the multiarmed bandit problem. Machine Learn. 47:235–256.Google Scholar
• Ayer T, Alagoz O, Stout NK (2012) OR forum—A POMDP approach to personalize mammography screening decisions. Oper. Res. 60(5):1019–1034.Link, Google Scholar
• Bai M, Storer RH, Tonkay GL (2022) Surgery sequencing coordination with recovery resource constraints. INFORMS J. Comput. 34(2):1207–1223.Link, Google Scholar
• Bartzokis G, Sultzer D, Lu PH, Nuechterlein KH, Mintz J, Cummings JL (2004) Heterogeneous age-related breakdown of white matter structural integrity: Implications for cortical “disconnection” in aging and Alzheimer’s disease. Neurobiology Aging 25(7):843–851.Google Scholar
• Besbes O, Gur Y, Zeevi A (2014) Stochastic multi-armed-bandit problem with non-stationary rewards. Adv. Neural Inform. Processing Systems, vol. 27 (Curran Associates, Inc., Red Hook, NY), 199–207.Google Scholar
• Biesanz JC, Deeb-Sossa N, Papadakis AA, Bollen KA, Curran PJ (2004) The role of coding time in estimating and interpreting growth curve models. Psych. Methods 9(1):30–52.Google Scholar
• Blennow K, Zetterberg H (2018) Biomarkers for Alzheimer’s disease: Current status and prospects for the future. J. Internal Medicine 284(6):643–663.Google Scholar
• Bouneffouf D, Claeys E (2020) Online hyper-parameter tuning for the contextual bandit. ICASSP 2021-2021 IEEE Internat. Conf. Acoustics Speech Signal Processing (ICASSP) (Toronto), 3445–3449.Google Scholar
• Bradley PS, Fayyad UM, Mangasarian OL (1999) Mathematical programming for data mining: Formulations and challenges. INFORMS J. Comput. 11(3):217–238.Link, Google Scholar
• Cesa-Bianchi N, Gentile C, Zappella G (2013) A gang of bandits. Adv. Neural Inform. Processing Systems (Curran Associates Inc., Red Hook, NY), 737–745.Google Scholar
• Chen S, Xie W (2022) On cluster-aware supervised learning: Frameworks, convergent algorithms, and applications. INFORMS J. Comput. 34(1):481–502.Link, Google Scholar
• Chen W, Wang Y, Yuan Y, Wang Q (2016) Combinatorial multi-armed bandit and its extension to probabilistically triggered arms. J. Machine Learn. Res. 17(1):1746–1778.Google Scholar
• DeSarbo WS, Cron WL (1988) A maximum likelihood methodology for clusterwise linear regression. J. Classification 5(2):249–282.Google Scholar
• Ding Q, Kang Y, Liu YW, Lee TCM, Hsieh CJ, Sharpnack J (2022) Syndicated bandits: A framework for auto tuning hyper-parameters in contextual bandit algorithms. Adv. Neural Inform. Processing Systems, vol. 35 (Curran Associates Inc., Red Hook, NY), 1170–1181.Google Scholar
• Dzhoha A, Rozora I (2023) Multi-armed bandit problem with online clustering as side information. J. Comput. Appl. Math. 427:115132.Google Scholar
• Filippi S, Cappe O, Garivier A, Szepesvári C (2010) Parametric bandits: The generalized linear case. Adv. Neural Inform. Processing Systems, vol. 23 (Curran Associates, Inc., Red Hook, NY), 586–594.Google Scholar
• Friansa K, Haq IN, Santi BM, Kurniadi D, Leksono E, Yuliarto B (2017) Development of battery monitoring system in smart microgrid based on internet of things (IoT). Procedia Engrg. 170:482–487.Google Scholar
• Garivier A, Moulines E (2011) On upper-confidence bound policies for switching bandit problems. Internat. Conf. Algorithmic Learn. Theory (Springer, Berlin, Heidelberg), 174–188.Google Scholar
• Gartner D, Kolisch R, Neill DB, Padman R (2015) Machine learning approaches for early DRG classification and resource allocation. INFORMS J. Comput. 27(4):718–734.Link, Google Scholar
• Gerdtham UG, Trivedi PK (2001) Equity in Swedish health care reconsidered: New results based on the finite mixture model. Health Econom. 10(6):565–572.Google Scholar
• Gómez AME, Li D, Paynabar K (2022) An adaptive sampling strategy for online monitoring and diagnosis of high-dimensional streaming data. Technometrics 64(2):253–269.Google Scholar
• Guo J, Yan H, Zhang C (2023) A Bayesian partially observable online change detection approach with Thompson sampling. Technometrics 65(2):179–191.Google Scholar
• Guo J, Yuan C, Shang N, Zheng T, Bello NA, Kiryluk K, Weng C, Wang S (2021) Similarity-based health risk prediction using domain fusion and electronic health records data. J. Biomedical Informatics 116:103711.Google Scholar
• Gyanwali B, Shaik MA, Tan BY, Venketasubramanian N, Chen C, Hilal S (2019) Risk factors for and clinical relevance of incident and progression of cerebral small vessel disease markers in an Asian memory clinic population. J. Alzheimer’s Disease 67(4):1209–1219.Google Scholar
• Han K, He Y, Liu AX, Tang S, Huang H (2020) Differentially private and budget-limited bandit learning over matroids. INFORMS J. Comput. 32(3):790–804.Link, Google Scholar
• Head D, Buckner RL, Shimony JS, Williams LE, Akbudak E, Conturo TE, McAvoy M, Morris JC, Snyder AZ (2004) Differential vulnerability of anterior white matter in nondemented aging with minimal acceleration in dementia of the Alzheimer type: Evidence from diffusion tensor imaging. Cerebral Cortex 14(4):410–423.Google Scholar
• Kamienny PA, Pirotta M, Lazaric A, Lavril T, Usunier N, Denoyer L (2020) Learning adaptive exploration strategies in dynamic environments through informed policy regularization. Preprint, submitted May 6, https://arxiv.org/abs/2005.02934.Google Scholar
• Kang Y, Hsieh CJ, Lee T (2024) Online continuous hyperparameter optimization for generalized linear contextual bandits. Trans. Machine Learn. Res. (OpenReview.net).Google Scholar
• Kumar A, Yao W, Chu CH (2013) Flexible process compliance with semantic constraints using mixed-integer programming. INFORMS J. Comput. 25(3):543–559.Link, Google Scholar
• Lattimore T, Crammer K, Szepesvári C (2015) Linear multi-resource allocation with semi-bandit feedback. Adv. Neural Inform. Processing Systems, vol. 28 (MIT Press, Cambridge, MA), 964–972.Google Scholar
• Lazarou I, Karakostas A, Stavropoulos TG, Tsompanidis T, Meditskos G, Kompatsiaris I, Tsolaki M (2016) A novel and intelligent home monitoring system for care support of elders with cognitive impairment. J. Alzheimer’s Disease 54(4):1561–1591.Google Scholar
• Li L, Chu W, Langford J, Schapire RE (2010) A contextual-bandit approach to personalized news article recommendation. Proc. 19th Internat. Conf. World Wide Web (Association for Computing Machinery, New York), 661–670.Google Scholar
• Liang Q, Cheng X, Huang SC, Chen D (2013) Opportunistic sensing in wireless sensor networks: Theory and application. IEEE Trans. Comput. 63(8):2002–2010.Google Scholar
• Lin Y, Liu S, Huang S (2018a) Selective sensing of a heterogeneous population of units with dynamic health conditions. IISE Trans. 50(12):1076–1088.Google Scholar
• Lin Y, Huang S, Simon GE, Liu S (2016) Analysis of depression trajectory patterns using collaborative learning. Math. Biosciences 282:191–203.Google Scholar
• Lin Y, Liu K, Byon E, Qian X, Liu S, Huang S (2018b) A collaborative learning framework for estimating many individualized regression models in a heterogeneous population. IEEE Trans. Reliability 67(1):328–341.Google Scholar
• Löwe B, Unützer J, Callahan CM, Perkins AJ, Kroenke K (2004) Monitoring depression treatment outcomes with the patient health questionnaire-9. Medical Care 42(12):1194–1201.Google Scholar
• Lu Y, Xu Z, Tewari A (2021) Bandit algorithms for precision medicine. Preprint, submitted August 10, https://arxiv.org/abs/2108.04782.Google Scholar
• Mate A, Perrault A, Tambe M (2021) Risk-aware interventions in public health: Planning with restless multi-armed bandits. AAMAS (International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC), 880–888.Google Scholar
• Mate A, Madaan L, Taneja A, Madhiwalla N, Verma S, Singh G, Hegde A, Varakantham P, Tambe M (2022) Field study in deploying restless multi-armed bandits: Assisting non-profits in improving maternal and child health. Proc. AAAI Conf. Artificial Intelligence 36(11):12017–12025.Google Scholar
• Meng Q, Xiao K, Shen D, Zhu H, Xiong H (2022) Fine-grained job salary benchmarking with a nonparametric dirichlet process–based latent factor model. INFORMS J. Comput. 34(5):2443–2463.Link, Google Scholar
• Mitchell AJ (2009) A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. J. Psychiatric Res. 43(4):411–431.Google Scholar
• Mouiha A, Duchesne S, Alzheimer’s Disease Neuroimaging Initiative (2012) Toward a dynamic biomarker model in Alzheimer’s disease. J. Alzheimer’s Disease 30(1):91–100.Google Scholar
• Mueller SG, Weiner MW, Thal LJ, Petersen RC, Jack C, Jagust W, Trojanowski JQ, Toga AW, Beckett L (2005) The Alzheimer’s disease neuroimaging initiative. Neuroimaging Clinics North Amer. 15(4):869–877.Google Scholar
• Nabhan M, Mei Y, Shi J (2021) Correlation-based dynamic sampling for online high dimensional process monitoring. J. Quality Tech. 53(3):289–308.Google Scholar
• Ouyang M, Tu D, Tong L, Sarwar M, Bhimaraj A, Li C, Cote GL, Di Carlo D (2021) A review of biosensor technologies for blood biomarkers toward monitoring cardiovascular diseases at the point-of-care. Biosensors Bioelectronics 171:112621.Google Scholar
• Owens DK, Davidson KW, Krist AH, Barry MJ, Cabana M, Caughey AB, Doubeni CA, et al. (2020) Screening for cognitive impairment in older adults: US preventive services task force recommendation statement. JAMA 323(8):757–763.Google Scholar
• Papakonstantinou KG, Shinozuka M (2014) Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part II: POMDP implementation. Reliability Engrg. System Safety 130:214–224.Google Scholar
• Pawar P, Jones V, Van Beijnum BJF, Hermens H (2012) A framework for the comparison of mobile patient monitoring systems. J. Biomedical Informatics 45(3):544–556.Google Scholar
• Petersen RC, Smith GE, Waring SC, Ivnik RJ, Tangalos EG, Kokmen E (1999) Mild cognitive impairment: Clinical characterization and outcome. Arch. Neurology 56(3):303–308.Google Scholar
• Petersen RC, Aisen PS, Beckett LA, Donohue MC, Gamst AC, Harvey DJ, Jack CR, et al. (2010) Alzheimer’s disease neuroimaging initiative (ADNI): Clinical characterization. Neurology 74(3):201–209.Google Scholar
• Peterson SJ, Luthans F, Avolio BJ, Walumbwa FO, Zhang Z (2011) Psychological capital and employee performance: A latent growth modeling approach. Personnel Psych. 64(2):427–450.Google Scholar
• Qin L, Chen S, Zhu X (2014) Contextual combinatorial bandit and its application on diversified online recommendation. Proc. 2014 SIAM Internat. Conf. Data Mining (SIAM, Philadelphia), 461–469.Google Scholar
• Rahimi-Eichi H, Ojha U, Baronti F, Chow MY (2013) Battery management system: An overview of its application in the smart grid and electric vehicles. IEEE Indust. Electronics Magazine 7(2):4–16.Google Scholar
• Sharif Z, Jung LT, Ayaz M, Yahya M, Pitafi S (2023) Priority-based task scheduling and resource allocation in edge computing for health monitoring system. J. King Saud Univ. Comput. Inform. Sci. 35(2):544–559.Google Scholar
• Streur MM, Ratcliffe SJ, Callans DJ, Shoemaker MB, Riegel BJ (2018) Atrial fibrillation symptom profiles associated with healthcare utilization: A latent class regression analysis. Pacing Clinical Electrophysiology 41(7):741–749.Google Scholar
• Su Y, Dong J, Sun J, Zhang Y, Ma S, Li M, Zhang A, et al. (2021) Cognitive function assessed by mini-mental state examination and risk of all-cause mortality: A community-based prospective cohort study. BMC Geriatrics 21(1):524.Google Scholar
• Szlam A, Tulloch A, Tygert M (2017) Accurate low-rank approximations via a few iterations of alternating least squares. SIAM J. Matrix Anal. Appl. 38(2):425–433.Google Scholar
• Tsai CW, Chiu HT, Huang HC, Ting IW, Yeh HC, Kuo CC (2018) Uric acid predicts adverse outcomes in chronic kidney disease: A novel insight from trajectory analyses. Nephrology Dialysis Transplantation 33(2):231–241.Google Scholar
• Uschmajew A (2012) Local convergence of the alternating least squares algorithm for canonical tensor approximation. SIAM J. Matrix Anal. Appl. 33(2):639–652.Google Scholar
• van der Meulen M, Dirven L, Bakunina K, van den Bent MJ, Issa S, Doorduijn JK, Bromberg JE (2021) MMSE is an independent prognostic factor for survival in primary central nervous system lymphoma. J. Neuro-Oncology 152:357–362.Google Scholar
• Vermunt JK, Magidson J (2002) Latent class cluster analysis. Applied Latent Class Analysis (Cambridge University Press, Cambridge, UK), 89–106.Google Scholar
• Wang J, Taaffe MR (2015) Multivariate mixtures of normal distributions: Properties, random vector generation, fitting, and as models of market daily changes. INFORMS J. Comput. 27(2):193–203.Link, Google Scholar
• Wang H, Wu Q, Wang H (2016) Learning hidden features for contextual bandits. Proc 25th ACM Internat. Conf. Inform. Knowledge Management (Association for Computing Machinery, New York), 1633–1642.Google Scholar
• Wang Q, Zeng C, Zhou W, Li T, Iyengar SS, Shwartz L, Grabarnik GY (2018) Online interactive collaborative filtering using multi-armed bandit with dependent arms. IEEE Trans. Knowledge Data Engrg. 31(8):1569–1580.Google Scholar
• Weimer DL, Sager MA (2009) Early identification and treatment of Alzheimer’s disease: Social and fiscal outcomes. Alzheimer’s Dementia 5(3):215–226.Google Scholar
• Wu Q, Wang H, Gu Q, Wang H (2016) Contextual bandits in a collaborative environment. Proc. 39th Internat. ACM SIGIR Conf. Res. Development Inform. Retrieval (Association for Computing Machinery, New York), 529–538.Google Scholar
• Yang J, Hu W, Lee JD, Du SS (2021) Impact of representation learning in linear bandits. Internat. Conf. Learn. Representations (ICLR, Appleton, WI).Google Scholar
• Yu F, Cui L, Chen H, Cao Y, Liu N, Huang W, Xu Y, Lu H (2022) Healthnet: A health progression network via heterogeneous medical information fusion. IEEE Trans. Neural Networks Learn. Systems 34(10):6940–6954.Google Scholar
• Zamaninasab Z, Najafipour H, Mirzaee M, Bahrampour A (2022) A cluster-wise linear regression model to investigate the effect of demographical and clinical variables on the average depression score. Medical J. Islamic Republic Iran 36:116.Google Scholar
• Zang W, Miao F, Gravina R, Sun F, Fortino G, Li Y (2020) CMDP-based intelligent transmission for wireless body area network in remote health monitoring. Neural Comput. Appl. 32:829–837.Google Scholar
• Zhou J, Liu J, Narayan VA, Ye J, Alzheimer’s Disease Neuroimaging Initiative (2013) Modeling disease progression via multi-task learning. NeuroImage 78:233–248.Google Scholar
• Zhu D, Wang L (2018) Exploring latent structures of Alzheimer’s disease via structure learning. 2018 IEEE 15th Internat. Sympos. Biomedical Imaging (IEEE, Piscataway, NJ), 536–540.Google Scholar
• Zhu F, Panwar B, Dodge HH, Li H, Hampstead BM, Albin RL, Paulson HL, Guan Y (2016) Compass: A computational model to predict changes in MMSE scores 24-months after initial assessment of Alzheimer’s disease. Sci. Rep. 6(1):34567.Google Scholar