Restless Bandit Marginal Productivity Indices, Diminishing Returns, and Optimal Control of Make-to-Order/Make-to-Stock M/G/1 Queues

References

• Ansell P. S., Glazebrook K. D., Niño-Mora J., O’Keeffe M. Whittle’s index policy for a multi-class queueing system with convex holding costs. Math. Methods Oper. Res. (2003) 57:21–39Crossref, Google Scholar
• Bell C. E. Characterization and computation of optimal policies for operating an M/G/1 queue with removable server. Oper. Res. (1971) 19:208–218Link, Google Scholar
• Bertsekas D. P.Dynamic Programming: Deterministic and Stochastic Models (1987) (Prentice-Hall, Englewood Cliffs, NJ) Google Scholar
• Bertsimas D., Niño-Mora J. Conservation laws, extended polymatroids and multiarmed bandit problems; A polyhedral approach to indexable systems. Math. Oper. Res. (1996) 21:257–306Link, Google Scholar
• Bertsimas D., Niño-Mora J. Restless bandits, linear programming relaxations, and a primal-dual index heuristic. Oper. Res. (2000) 48:80–90Link, Google Scholar
• Blackwell D. Discrete dynamic programming. Ann. Math. Statist. (1962) 33:719–726Crossref, Google Scholar
• Buzacott J. A., Shanthikumar J. G.Stochastic Models of Manufacturing Systems (1993) (Prentice-Hall, Englewood Cliffs, NJ) Google Scholar
• Coffman E. G., Mitrani I. A characterization of waiting time performance realizable by single-server queues. Oper. Res. (1980) 28:810–821Link, Google Scholar
• Cox D. R., Smith W. L.Queues (1961) (Methuen, London, UK) Google Scholar
• Dacre M., Glazebrook K., Niño-Mora J. The achievable region approach to the optimal control of stochastic systems. J. Roy. Statist. Soc., Ser. B, Statist. Methodology (1999) 61:747–791Crossref, Google Scholar
• D’Epénoux F. Sur un problème de production et de stockage dans l’aléatoire. RAIRO Rech. Opér. (1960) 14:3–16[English translation. 1963. A probabilistic production and inventory problem. Management Sci. 10 98–108]Google Scholar
• De Véricourt F., Karaesmen F., Dallery Y. Dynamic scheduling in a make-to-stock system: A partial characterization of optimal policies. Oper. Res. (2000) 48:811–819Link, Google Scholar
• Dusonchet F., Hongler M.-O. Continuous-time restless bandit and dynamic scheduling for make-to-stock production. IEEE Trans. Robotics Automation (2003) 19:977–990Crossref, Google Scholar
• Edmonds J. Matroids and the greedy algorithm. Math. Programming (1971) 1:127–136Crossref, Google Scholar
• Federgruen A., Groenevelt H. Characterization and optimization of achievable performance in general queueing systems. Oper. Res. (1988) 36:733–741Link, Google Scholar
• Feinberg E. A., Shwartz A., Feinberg E. A., Shwartz A. Mixed criteria. Handbook of Markov Decision Processes: Methods and Applications (2002) (Kluwer, Boston, MA) 209–230Crossref, Google Scholar
• Gittins J. C. Bandit processes and dynamic allocation indices. J. Roy. Statist. Soc., Ser. B (1979) 41:148–177Google Scholar
• Gittins J. C., Jones D. M. A dynamic allocation index for the sequential design of experiments. Progress in Statistics, European Meeting of Statisticians, Budapest, 1972 (1974) (North-Holland, Amsterdam, The Netherlands) 241–266Colloquia Math. Soc. János Bolyai, Vol. 9Google Scholar
• Glazebrook K. D., Niño-Mora J. Parallel scheduling of multiclass M/M/m queues: Approximate and heavy-traffic optimization of achievable performance. Oper. Res. (2001) 49:609–623Link, Google Scholar
• Ha A. Y. Optimal dynamic scheduling policy for a make-to-stock production system. Oper. Res. (1997) 45:42–53Link, Google Scholar
• Kantorovich L. V.Ekonomicheskii Raschet Nailuchshego Ispolzovania Resursov (1959) (Academy of Sciences USSR). [English translation. 1965. The Best Use of Economic Resources. Harvard University Press, Cambridge, MA]Google Scholar
• Kleinrock L. A conservation law for a wide class of queueing disciplines. Naval Res. Logist. Quart. (1965) 12:181–192Crossref, Google Scholar
• Kleinrock L.Queueing Systems (1975) Vol. I(Wiley, New York) . TheoryGoogle Scholar
• Klimov G. P. Time-sharing service systems. I. Theory Probab. Appl. (1974) 19:532–551Crossref, Google Scholar
• Koopmans T. C.Three Essays on the State of Economic Science (1957) (McGraw-Hill, New York) . Chapter on Allocation of resources and the price systemGoogle Scholar
• Lewis M. E., Puterman M. L., Feinberg E., Schwartz A. Bias optimality. Handbook of Markov Decision Processes (2002) (Kluwer, Boston, MA) 89–111Crossref, Google Scholar
• Manne A. S. Linear programming and sequential decisions. Management Sci. (1960) 6:259–267Link, Google Scholar
• Marshall K. T., Wolff R. W. Customer average and time average queue lengths and waiting times. J. Appl. Probab. (1971) 8:535–542Crossref, Google Scholar
• Morse P. M.Queues, Inventories and Maintenance: The Analysis of Operational Systems with Variable Demand and Supply (1958) (Wiley, New York) Crossref, Google Scholar
• Niño-Mora J. Restless bandit index heuristics for multiclass queueing network control. Conf. Stochastic Networks (2000) (University of Wisconsin-Madison, Madison, WI) . PresentationGoogle Scholar
• Niño-Mora J. Restless bandits: PCL-indexability conditions and queueing control applications. 27th Conf. Stochastic Processes and Their Applications (2001) Cambridge, UKPresentationGoogle Scholar
• Niño-Mora J. Restless bandits, partial conservation laws and indexability. Adv. Appl. Probab. (2001) 33:76–98Crossref, Google Scholar
• Niño-Mora J., Floudas C. A., Pardalos P. M. Stochastic scheduling. Encyclopedia of Optimization (2001) Vol. 5(Kluwer, Dordrecht, The Netherlands) 367–372Google Scholar
• Niño-Mora J. Dynamic allocation indices for restless projects and queueing admission control: A polyhedral approach. Math. Programming (2002) 93:361–413Crossref, Google Scholar
• Niño-Mora J., Srikant R., Veeravalli V. V. Restless bandit marginal productivity indices, diminishing returns and scheduling a multiclass make-to-order/-stock queue. Proc. 41st Annual Allerton Conf. Comm., Control Comput. (2003) Monticello, IL:100–109Google Scholar
• Niño-Mora J. Restless bandit dynamic allocation indices II: Multi-project case and dynamic scheduling of a multiclass make-to-order/-stock M/G/1 queue. (2004) . Working Paper 04-09, Department of Statistics, Universidad Carlos III de Madrid, Spain, FebruaryGoogle Scholar
• Papadimitriou C. H., Tsitsiklis J. N. The complexity of optimal queuing network control. Math. Oper. Res. (1999) 24:293–305Link, Google Scholar
• Peña-Pérez A., Zipkin P. Dynamic scheduling rules for a multiproduct make-to-stock queue. Oper. Res. (1997) 45:919–930Link, Google Scholar
• Puterman M. L.Markov Decision Processes: Discrete Stochastic Dynamic Programming (1994) (Wiley, New York) Crossref, Google Scholar
• Ross S. M. Average cost semi-Markov decision processes. J. Appl. Probab. (1970) 7:649–656Crossref, Google Scholar
• Shanthikumar J. G., Yao D. D. Multiclass queueing systems: Polymatroidal structure and optimal scheduling control. Oper. Res. (1992) 40:S293–S299Link, Google Scholar
• Tsoucas P. The region of achievable performance in a model of Klimov. (1991) . Tech. Report RC16543, IBM T. J. Watson Research Center, Yorktown Heights, New YorkGoogle Scholar
• Veatch M. H., Wein L. M. Scheduling a multiclass make-to-stock queue: Index policies and hedging points. Oper. Res. (1996) 44:634–647Link, Google Scholar
• Wein L. M. Dynamic scheduling of a multiclass make-to-stock queue. Oper. Res. (1992) 40:724–735Link, Google Scholar
• Weiss G. Branching bandit processes. Probab. Engrg. Inform. Sci. (1988) 2:269–278Crossref, Google Scholar
• Weitzman M. L. An “economics proof” of the supporting hyperplane theorem. Econom. Lett. (2000) 68:1–6Crossref, Google Scholar
• Whittle P., Gani J. Restless bandits: Activity allocation in a changing world. A Celebration of Applied Probability, J. Appl. Probab. Special (1988) Vol. 25A(Applied Probability Trust, Sheffield, UK) 287–298Google Scholar
• Whittle P.Optimal Control: Basics and Beyond (1996) (Wiley, Chichester, UK) Google Scholar