A Restless Bandit Model for Resource Allocation, Competition, and Reservation

References

• Avrachenkov KE, Borkar VS (2016) Whittle index policy for crawling ephemeral content. IEEE Trans. Control Network System 5(1):446–455.Crossref, Google Scholar
• Bäuerle N (2002) Optimal control of queueing networks: An approach via fluid models. Adv. Appl. Probabilities 34(2):313–328.Crossref, Google Scholar
• Bäuerle N (2000) Asymptotic optimality of tracking policies in stochastic networks. Ann. Appl. Probab. 1081(2-3):1065–1083.Crossref, Google Scholar
• Bertsimas D, Niño-Mora J (1996) Conservation laws, extended polymatroids and multiarmed bandit problems: A polyhedral approach to indexable systems. Math. Oper. Res. 21(2):257–306.Link, Google Scholar
• Bertsimas D, Nasrabadi E, Paschalidis ICGoogle Scholar (2015Google Scholar) Robust fluid processing networks.Google ScholarIEEE Trans. Automated ControlsGoogle Scholar 60Google Scholar(3Google Scholar):715Google Scholar–728Google Scholar.Google Scholar
• Google ScholarCezik MT, L’Ecuyer PGoogle Scholar (2008Google Scholar) Staffing multiskill call centers via linear programming and simulationGoogle Scholar. Management Sci.Google Scholar 54Google Scholar(2Google Scholar):310Google Scholar–323Google Scholar.Google Scholar
• Chen X, Han Z, Zhang H, Xue G, Xiao Y, Bennis M (2017) Wireless resource scheduling in virtualized radio access networks using stochastic learning. IEEE Trans. Mobile Comput. 17(4):961–974.Crossref, Google Scholar
• Coddington EA, Levinson N (1955) Theory of Ordinary Differential Equations (Tata McGraw-Hill Education, McGraw-Hill, New York).Google Scholar
• Esposito F, Di Paola D, Matta I (2016) On distributed virtual network embedding with guarantees. IEEE/ACM Trans. Networks 24(1):569–582.Crossref, Google Scholar
• Fikar C, Hirsch P (2017) Home healthcare routing and scheduling: A review. Comput. Oper. Res. 77:86–95.Crossref, Google Scholar
• Freidlin MI, Wentzell AD, Szücs J (2012) Random Perturbations of Dynamical Systems (Springer Science & Business Media, Berlin).Google Scholar
• Fu J (2016) Energy-efficient heuristics for job assignment in server farms. PhD thesis, Department of Electronic Engineering, City University of Hong Kong, Hong Kong.Google Scholar
• Fu J, Moran B, Guo J, Wong EWM, Zukerman M (2016) Asymptotically optimal job assignment for energy-efficient processor-sharing server farms. IEEE J. Selected Areas Comm. 34(12):4008–4023.Crossref, Google Scholar
• Gittins JC (1979) Bandit processes and dynamic allocation indices. J. Royal Statist. Soc. B 41(2):148–177.Crossref, Google Scholar
• Gittins JC, Jones DM (1974) A dynamic allocation index for the sequential design of experiments. Gani J, ed. Progress in Statistics (North-Holland, Amsterdam), 241–266.Google Scholar
• Gittins JC, Glazebrook K, Weber RR (2011) Multi-Armed Bandit Allocation Indices, 2nd ed. (Wiley, New York).Crossref, Google Scholar
• Kelly FP (1991) Loss networks. Ann. Appl. Probab. 1(3):319–378.Crossref, Google Scholar
• Krishnamurthy V, Djonin DV (2007) Structured threshold policies for dynamic sensor scheduling: A partially observed Markovdecision process approach. IEEE Trans. Signal Processing 55(10):4938–4957.Crossref, Google Scholar
• Larrañaga M, Ayesta U, Verloop IM (2015) Asymptotically optimal index policies for an abandonment queue with convex holding cost. Queueing Systems 81(2-3):99–169.Crossref, Google Scholar
• Le Ny J, Feron E, Dahleh MA (2010) Scheduling continuous-time Kalman filters. IEEE Trans. Automated Controls 56(6):1381–1394.Crossref, Google Scholar
• Lieder A, Moeke D, Koole G, Stolletz R (2015) Task scheduling in long-term care facilities: A client-centered approach. Oper. Res. Health Care 6:11–17.Crossref, Google Scholar
• Liu H, Liu K, Zhao Q (2012) Learning in a changing world: Restless multiarmed bandit with unknown dynamics. IEEE Trans. Inform. Theory 59(3):1902–1916.Crossref, Google Scholar
• Meyn S (2008) Control Techniques for Complex Networks (Cambridge University Press, Cambridge, UK).Google Scholar
• Nazarathy Y, Weiss G (2009) Near optimal control of queueing networks over a finite time horizon. Ann. Oper. Res. 170(1):233–249.Crossref, Google Scholar
• Niño-Mora J (2001) Restless bandits, partial conservation laws and indexability. Adv. Appl. Probabilities 33(1):76–98.Crossref, Google Scholar
• Niño-Mora J (2002) Dynamic allocation indices for restless projects and queueing admission control: A polyhedral approach. Math. Programming 93(3):361–413.Crossref, Google Scholar
• Niño-Mora J (2007) Dynamic priority allocation via restless bandit marginal productivity indices. TOP 15(2):161–198.Crossref, Google Scholar
• Niño-Mora J (2012) Admission and routing of soft real-time jobs to multiclusters: Design and comparison of index policies. Comput. Oper. Res. 39(12):3431–3444.Crossref, Google Scholar
• Niño-Mora J (2019) Resource allocation and routing in parallel multi-server queues with abandonments for cloud profit maximization. Comput. Oper. Res. 103:221–236.Crossref, Google Scholar
• Papadimitriou CH, Tsitsiklis JN (1999) The complexity of optimal queuing network control. Math. Oper. Res. 24(2):293–305.Link, Google Scholar
• Ross SM (1992) Applied Probability Models with Optimization Applications (Dover Publications, New York).Google Scholar
• Serfozo R (2009) Basics of Applied Stochastic Processes (Springer Science & Business Media, Berlin).Crossref, Google Scholar
• Stolyar AL (2004) Maxweight scheduling in a generalized switch: State space collapse and workload minimization in heavy traffic. Ann. Appl. Probabilities 14(1):1–53.Crossref, Google Scholar
• Stolyar AL (2013) An infinite server system with general packing constraints. Oper. Res. 61(5):1200–1217.Link, Google Scholar
• Stolyar AL (2017) Large-scale heterogeneous service systems with general packing constraints. Adv. Appl. Probabilities 49(1):61–83.Crossref, Google Scholar
• Verloop IM (2016) Asymptotically optimal priority policies for indexable and non-indexable restless bandits. Ann. Appl. Probabilities 26(4):1947–1995.Crossref, Google Scholar
• Wallace RB, Whitt W (2004) Resource pooling and staffing in call centers with skill-based routing. Oper. Res. 7(4):276–294.Google Scholar
• Wang J, Ren X, Mo Y, Shi L (2019) Whittle index policy for dynamic multi-channel allocation in remote state estimation. IEEE Trans. Automatic Control 99:1.Google Scholar
• Weber RR, Weiss G (1990) On an index policy for restless bandits. J. Appl. Probabilities 3:637–648.Crossref, Google Scholar
• Wei X, Neely MJ (2017) Data center server provision: Distributed asynchronous control for coupled renewal systems. IEEE/ACM Trans. Networks 25(4):2180–2194.Crossref, Google Scholar
• Whittle P (1988) Restless bandits: Activity allocation in a changing world. J. Appl. Probabilities 25:287–298.Crossref, Google Scholar