Computing a Classic Index for Finite-Horizon Bandits

References

• Bellman R. A problem in the sequential design of experiments. Sankhyā (1956) 16(3/4):221–229Google Scholar
• Berry D. A., Fristedt B.Bandit Problems: Sequential Allocation of Experiments (1985) (Chapman & Hall, London) Crossref, Google Scholar
• Bradt R. N., Johnson S. M., Karlin S. On sequential designs for maximizing the sum of n observations. Ann. Math. Statist. (1956) 27(4):1060–1074Crossref, Google Scholar
• Caro F., Gallien J. Dynamic assortment with demand learning for seasonal consumer goods. Management Sci. (2007) 53(2):276–292Link, Google Scholar
• Dongarra J. J., Eijkhout V. Numerical linear algebra algorithms and software. J. Comput. Appl. Math. (2000) 123(1–2):489–514Crossref, Google Scholar
• Ginebra J., Clayton M. K. Small-sample performance of Bernoulli two-armed bandit Bayesian strategies. J. Statist. Plann. Inference (1999) 79(1):107–122Crossref, Google Scholar
• Gittins J. C. Bandit processes and dynamic allocation indices. J. Roy. Statist. Soc. Ser. B (1979) 41(2):148–177Google Scholar
• Gittins J. C.Multi-Armed Bandit Allocation Indices (1989) (John Wiley & Sons, Chichester, UK) Google Scholar
• Gittins J. C., Jones D. M., Gani J., Sarkadi K., Vincze I. A dynamic allocation index for the sequential design of experiments. Progress in Statistics (European Meeting of Statisticians, Budapest, 1972) (1974) (North-Holland, Amsterdam) 241–266Google Scholar
• Niño-Mora J. Restless bandits, partial conservation laws and indexability. Adv. Appl. Probab. (2001) 33(1):76–98Crossref, Google Scholar
• Niño-Mora J. Dynamic allocation indices for restless projects and queueing admission control: A polyhedral approach. Math. Programming (2002) 93(3):361–413Crossref, Google Scholar
• Niño-Mora J. A marginal productivity index policy for the finite-horizon multiarmed bandit problem. CDC-ECC '05: Proc. 44th IEEE Conf. Decision Control Eur. Control Conf. 2005 (2005) Seville, Spain(IEEE, Washington, DC) 1718–1722Crossref, Google Scholar
• Niño-Mora J. A (2/3) n3 fast-pivoting algorithm for the Gittins index and optimal stopping of a Markov chain. INFORMS J. Comput. (2007) 19(4):596–606Link, Google Scholar
• Niño-Mora J. Computing an index policy for multiarmed bandits with deadlines. ValueTools '08: Proc. Third Internat. Conf. Performance Evaluation Methodologies Tools (2008) Athens, Greece(ICST, Brussels) . Article 13Crossref, Google Scholar
• Robbins H. Some aspects of the sequential design of experiments. Bull. Amer. Math. Soc. (1952) 58(5):527–535Crossref, Google Scholar
• Varaiya P. P., Walrand J. C., Buyukkoc C. Extensions of the multiarmed bandit problem: The discounted case. IEEE Trans. Automat. Control (1985) 30(5):426–439Crossref, Google Scholar
• Wang Y.-G. Error bounds for calculation of the Gittins indices. Austral. J. Statist. (1997) 39(2):225–233Crossref, Google Scholar