Markov Decision Processes with Slow Scale Periodic Decisions

References

• Bather J. Optimal decision procedures for finite Markov chains III. Adv. Appl. Probab. (1973) 5:541–553Crossref, Google Scholar
• Bäuerle N. Discounted stochastic fluid programs. Math. Oper. Res. (2001) 26:401–420Link, Google Scholar
• Davis M. H. A.Markov Models and Optimization (1993) (Chapman and Hall, London, U.K.) Crossref, Google Scholar
• Delebecque F., Quadrat J. P. Optimal control of Markov chains admitting strong and weak interactions. Automatica (1981) 17:281–296Crossref, Google Scholar
• Hinderer K., Hubner G., Tijms H., Wessels J. On approximate and exact solution for finite stage dynamic programs. Markov Decision Theory (1977) (Mathematical Centre, Amsterdam, The Netherlands) Google Scholar
• Jacobson M. Asymptotic properties of two timescale Markov decision processes. (1998) . M.Sc. thesis, Technion, Israel Institute of Technology, Haifa, Israel. (Available at http://www.ee.technion.ac.il/∼adam/PAPERS/MWJ-MSc.ps)Google Scholar
• Phillips R. G., Kokotovic P. A singularly perturbation approach to modeling and control of Markov chains. IEEE Trans. Automatic Control (1981) AC-26:1087–1094Crossref, Google Scholar
• Puterman M. L.Markov Decision Processes: Discrete Stochastic Dynamic Programming (1994) (John Wiley & Sons, Inc., New York) Crossref, Google Scholar
• Schweitzer P. J. A value iteration scheme for undiscounted multichain markov renewal programs. Zeitschrift Oper. Res. (1984) 28:143–152Crossref, Google Scholar
• Schweitzer P. J., Federgruen A. The asymptotic behavior of undiscounted value iteration in Markov decision problems. Math. Oper. Res. (1977) 2:360–382Link, Google Scholar
• Schweitzer P. J., Federgruen A. The functional equations of undiscounted Markov renewal programming. Math. Oper. Res. (1978) 3:308–321Link, Google Scholar
• Schweitzer P. J., Federgruen A. Geometric convergence of value iteration in multichain Markov decision processes. Adv. Appl. Probab. (1979) 11:188–217Crossref, Google Scholar
• Sethi S. P., Zhang Q.Hierarchical Decision Making in Stochastic Manufacturing Systems (1994) (Birkhäuser, Boston, MA) Crossref, Google Scholar
• Shapiro J. Turnpike planning horizons for a Markovian decision model. Management Sci. (1968) 14:292–300Link, Google Scholar