Tuning Approximate Dynamic Programming Policies for Ambulance Redeployment via Direct Search

References

• Adelman, D. A price-directed approach to stochastic inventory routing. Operations Research, 52(4):499–514, 2004. MR2075790Link, Google Scholar
• Adelman, D. Dynamic bid-prices in revenue management. Operations Research, 55(4):647–661, 2007. MR2349028Link, Google Scholar
• Alanis, R., Ingolfsson, A., and Kolfal, B. A Markov chain model for an EMS system with repositioning, 2010.Google Scholar
• Andersson, T. Decision support tools for dynamic fleet management. PhD thesis, Department of Science and Technology, Linköepings Universitet, Norrköeping, Sweden, 2005.Google Scholar
• Andersson, T. and Vaerband, P. Decision support tools for ambulance dispatch and relocation. Journal of the Operational Research Society, 58:195–201, 2007.Google Scholar
• Bandar, D., Mayorga, M. E., and McLay, L. A. Optimal dispatching strategies for emergency vehicles to increase patient survivability. To appear, International Journal of Operational Research, 2012. MR3014423Google Scholar
• Berman, O. Dynamic repositioning of indistinguishable service units on transportation networks. Transportation Science, 15(2), 1981. MR0639599Google Scholar
• Berman, O. Repositioning of distinguishable urban service units on networks. Computers and Operations Research, 8:105–118, 1981.Google Scholar
• Berman, O. Repositioning of two distinguishable service vehicles on networks. IEEE Transactions on Systems, Man, and Cybernetics, SMC-11(3), 1981. MR0614679Google Scholar
• Bertsekas, D. Dynamic Programming and Optimal Control. Athena Scientific, Nashua, NH, 2005. MR2183196Google Scholar
• Bertsekas, D. and Shreve, S. Stochastic Optimal Control: The Discrete Time Case. Academic Press, New York, 1978. MR0511544Google Scholar
• Bertsekas, D. and Tsitsiklis, J. Neuro-Dynamic Programming. Athena Scientific, Belmont, Massachusetts, 1996.Google Scholar
• Boyan, J. A. Technical update: Least-squares temporal difference learning. Machine Learning, 49(2):233–246, 2002.Google Scholar
• Bradtke, S. J., Barto, A. G., and Kaelbling, P. Linear least-squares algorithms for temporal difference learning. In Machine Learning, pages 22–33, 1996.Google Scholar
• de Farias, D. P. and Van Roy, B. The linear programming approach to approximate dynamic programming. Operations Research, 51:2003, 2001. MR2019651Google Scholar
• Deng, G. and Ferris, M. C. Adaptation of the UOBYQA algorithm for noisy functions. In WSC ’06: Proceedings of the 38th conference on Winter simulation, pages 312–319. Winter Simulation Conference, 2006.Google Scholar
• Desai, V. V., Farias, V. F., and Moallemi, C. C. Approximate dynamic programming via a smoothed linear program. To appear in Operations Research, 2012. MR2995921Google Scholar
• Erkut, E., Ingolfsson, A., and Erdoğan, G. Ambulance deployment for maximum survival. Naval Research Logistics, 55(1):42–58, 2007. MR2378248Google Scholar
• Farias, V. F. and Van Roy, B. An approximate dynamic programming approach to network revenue management. Technical report, Stanford University, Department of Electrical Engineering, 2007.Google Scholar
• Gendreau, M., Laporte, G., and Semet, S. A dynamic model and parallel tabu search heuristic for real time ambulance relocation. Parallel Computing, 27:1641–1653, 2001.Google Scholar
• Gendreau, M., Laporte, G., and Semet, S. The maximal expected coverage relocation problem for emergency vehicles. Journal of the Operational Research Society, 57:22–28, 2006.Google Scholar
• Glynn, P. W. A GSMP formalism for discrete event systems. Proceedings of the IEEE, 77(1), 1989.Google Scholar
• Kolesar, P. and Walker, W. E. An algorithm for the dynamic relocation of fire companies. Operations Research, 22(2):249–274, 1974.Link, Google Scholar
• L’Ecuyer, P., Simard, R., Chen, E. J., and Kelton, W. D. An object-oriented random-number package with many long streams and substreams. Operations Research, 50(6):1073–1075, 2002.Link, Google Scholar
• Longstaff, F. A. and Schwartz, E. S. Valuing American options by simulation: A simple least-squares approach. The Review of Financial Studies, 14(1):113–147, 2001.Google Scholar
• Maxwell, M. S., Henderson, S. G., and Topaloglu, H. Ambulance redeployment: An approximate dynamic programming approach. In M. D. Rossetti, R. R. Hill, B. Johansson, A. Dunkin, and R. G. Ingalls, editors, Proceedings of the 2009 Winter Simulation Conference, pages 1850–1860, Piscataway, New Jersey, 2009. Institute of Electrical and Electronics Engineers, Inc.Google Scholar
• Maxwell, M. S., Henderson, S. G., and Topaloglu, H. Identifying effective policies in approximate dynamic programming: Beyond regression. In B. Johansson, S. Jain, J. Montoya-Torres, J. Hugan, and E. Yücesan, editors, Proceedings of the 2010 Winter Simulation Conference, Piscataway, New Jersey, 2010. Institute of Electrical and Electronics Engineers, Inc.Google Scholar
• Maxwell, M. S., Restrepo, M., Henderson, S. G., and Topaloglu, H. Approximate dynamic programming for ambulance redeployment. INFORMS Journal on Computing, 22(2):266–281, 2010. MR2677214Link, Google Scholar
• Nair, R. and Miller-Hooks, E. Evaluation of relocation strategies for emergency medical service vehicles. Transportation Research Record: Journal of the Transportation Research Board, 2137:63–73, 2009.Google Scholar
• Nelder, J. and Mead, R. A simplex method for function minimization. The Computer Journal, 7(4):308–313, 1965.Google Scholar
• Papadaki, K. and Powell, W. B. An adaptive dynamic programming algorithm for a stochastic multiproduct batch dispatch problem. Naval Research Logistics, 50(7):742–769, 2003. MR2001023Google Scholar
• Powell, M. UOBYQA: Unconstrained optimization by quadratic approximation. Mathematical Programming, 92(3):555–582, 2002. MR1905766Google Scholar
• Powell, W. B. Approximate Dynamic Programming: Solving the Curses of Dimensionality. John Wiley & Sons, Hoboken, NJ, 2007. MR2347698Google Scholar
• Puterman, M. L. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Hoboken, NJ, 2005.Google Scholar
• Richards, D. Optimised Ambulance Redeployment Strategies. PhD thesis, The University of Auckland, Auckland, New Zealand, 2007.Google Scholar
• Schmid, V. Solving the dynamic ambulance relocation and dispatching problem using approximate dynamic programming. European Journal of Operational Research, 219:611–621, 2012. MR2898941Google Scholar
• Schweitzer, P. J. and Seidmann, A. Generalized polynomial approximations in Markovian decision processes. Journal of Mathematical Analysis and Applications, 110(2):568–582, 1985. MR0805277Google Scholar
• Shapiro, A. Monte Carlo sampling methods. In A. Ruszczynski and A. Shapiro, editors, Stochastic Programming, volume 10 of Handbooks in Operations Research and Management Science, pages 353–425. Elsevier, 2003. MR2052758Google Scholar
• Spivey, M. Z. and Powell, W. B. The dynamic assignment problem. Transportation Science, 38(4):399–419, 2004.Link, Google Scholar
• Sutton, R. S. Learning to predict by the methods of temporal differences. In Machine Learning, pages 9–44. Kluwer Academic Publishers, 1988.Google Scholar
• Sutton, R. S. and Barto, A. G. Reinforcement Learning. The MIT Press, Cambridge, MA, 1998.Google Scholar
• Szita, I. and Lörincz, A. Learning tetris using the noisy cross-entropy method. Neural Computation., 18(12):2936–2941, 2006.Google Scholar
• Tesauro, G. TD-Gammon, a self-teaching backgammon program, achieves master-level play. Neural Computation, 6(2):215–219, 1994.Google Scholar
• Topaloglu, H. and Powell, W. B. Dynamic programming approximations for stochastic, time-staged integer multicommodity flow problems. INFORMS Journal on Computing, 18(1):31–42, 2006. MR2205745Link, Google Scholar
• Tsitsiklis, J. and Van Roy, B. Regression methods for pricing complex American-style options. IEEE Transactions on Neural Networks, 12(4):694–703, 2001.Google Scholar
• Van Roy, B., Bertsekas, D. P., Lee, Y., and Tsitsiklis, J. N. A neuro dynamic programming approach to retailer inventory management. In Proceedings of the IEEE Conference on Decision and Control, 1997.Google Scholar
• Zhang, L. Simulation Optimisation and Markov Models for Dynamic Ambulance Redeployment. PhD thesis, The University of Auckland, Auckland, New Zealand, 2012.Google Scholar
• Zhang, L., Mason, A., and Philpott, A. Optimization of a single ambulance move up. Preprint, 2010.Google Scholar