Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish Spaces

References

• Anderson W. J.Continuous-Time Markov Chains (1991) (Springer-Verlag, New York) Crossref, Google Scholar
• Bailey N. T. J.The Mathematical Theory of Infectious Diseases (1975) 2nd ed.(Griffin, London, UK) Google Scholar
• Bellman R.Dynamic Programming (1957) (Princeton University Press, Princeton, NJ) Google Scholar
• Chen M. F.From Markov Chains to Non-Equilibrium Particle Systems (2004) 2nd ed.(World Scientific Publishing Co. Inc., River Edge, NJ) Google Scholar
• Doshi B. T. Continuous-time control of Markov processes on an arbitrary state space: Discounted rewards. Ann. Statist. (1976) 4:1219–1235Crossref, Google Scholar
• Feinberg E. A. Continuous-time jump Markov decision processes: A discrete-event approach. Math. Oper. Res. (2004) 29:492–524Link, Google Scholar
• Feller W. On the integro-differential equations of purely discontinuous Markoff processes. Trans. Amer. Math. Soc. (1940) 48:488–515Crossref, Google Scholar
• Fleming W. H., Soner H. M.Controlled Markov Processes and Viscosity Solutions (1993) (Springer-Verlag, Berlin, Germany) Google Scholar
• Gihman I. I., Skorohod A. V.Controlled Stochastic Processes (1979) (Springer-Verlag, New York, Heidelberg, Berlin, Germany) Crossref, Google Scholar
• Guo X. P., Cao X.-R. Optimal control of ergodic continuous-time Markov chains with average sample-path rewards. SIAM J. Control Optim. (2005) 44:29–48Crossref, Google Scholar
• Guo X. P., Hernández-Lerma O. Continuous-time controlled Markov chains. Ann. Appl. Probab. (2003) 13:363–388Crossref, Google Scholar
• Guo X. P., Hernández-Lerma O. Continuous-time controlled Markov chains with discounted rewards. Acta Appl. Math. (2003) 79:195–216Crossref, Google Scholar
• Guo X. P., Liu K. A note on optimality conditions for continuous-time Markov decision processes with average cost criterion. IEEE Trans. Automat. Control (2001) 46:1984–1989Crossref, Google Scholar
• Guo X. P., Zhu W. P. Denumerable state continuous-time Markov decision processes with unbounded cost and transition rates under the discounted criterion. J. Appl. Probab. (2002) 39:233–250Crossref, Google Scholar
• Haviv M., Puterman M. L. Bias optimality in controlled queuing systems. J. Appl. Probab. (1998) 35:136–150Crossref, Google Scholar
• Hernández-Lerma O., Govindan T. E. Nonstationary continuous-time Markov control processes with discounted costs on infinite horizon. Acta Appl. Math. (2001) 67:277–293Crossref, Google Scholar
• Hernández-Lerma O., Lasserre J. B.Further Topics on Discrete-Time Markov Control Processes (1999) (Springer-Verlag, New York) Crossref, Google Scholar
• Hitchcock S. E. Extinction probabilities in predator-prey models. J. Appl. Probab. (1986) 23:1–13Crossref, Google Scholar
• Holley R., Liggett T. M. Generalized Potlach and smoothing processes. Z. Wahrsch. Verw. Gebiete (1981) 55:165–195Crossref, Google Scholar
• Howard R. A.Dynamic Programming and Markov Processes (1960) (Wiley, New York) Google Scholar
• Kakumanu P. Continuously discounted Markov decision models with countable state and action spaces. Ann. Math. Statist. (1971) 42:919–926Crossref, Google Scholar
• Lefèvre C. Optimal control of a birth and death epidemic process. Oper. Res. (1981) 29:971–982Link, Google Scholar
• Lembersky M. R. On maximal rewards and ϵ-optimal policies in continuous time Markov chains. Ann. Statist. (1974) 2:159–169Crossref, Google Scholar
• Lewis M. E., Puterman M. L. A note on bias optimality in controlled queueing systems. J. Appl. Probab. (2000) 37:300–305Crossref, Google Scholar
• Lewis M. E., Puterman M. L. A probabilistic analysis of bias optimality in unichain Markov decision processes. IEEE Trans. Automat. Control (2001) 46:96–100Crossref, Google Scholar
• Liggett T. M., Spitzer F. Ergodic theorems for coupled random walks and other systems with locally interacting components. Z. Wahrsch. Verw. Gebiete (1981) 56:443–468Crossref, Google Scholar
• Lippman S. A. Applying a new device in the optimization of exponential queueing system. Oper. Res. (1975) 23:667–710Link, Google Scholar
• Lund R. B., Meyn S. P., Tweedie R. L. Computable exponential convergence rates for stochastically ordered Markov processes. Ann. Appl. Probab. (1996) 6:218–237Crossref, Google Scholar
• Meyn S. P., Tweedie R. L. Stability of Markovian processes III: Foster-Lyapunov criteria for continuous-time processes. Adv. Appl. Probab. (1993) 25:518–548Crossref, Google Scholar
• Miller R. L. Finite state continuous time Markov decision processes with an infinite planning horizon. J. Math. Anal. Appl. (1968) 22:552–569Crossref, Google Scholar
• Puterman M. L.Markov Decision Processes (1994) (Wiley, New York) Crossref, Google Scholar
• Rajarshi M. B. Simple proofs of two threshold theorems for a general stochastic epidemic. J. Appl. Probab. (1981) 18:721–724Crossref, Google Scholar
• Reuter G. E. H. Competition processes. Proc. Fourth Berkeley Sympos. Math. Statist. Probab. (1961) 2:421–430Google Scholar
• Ridler-Rowe C. J. Extinction times for certain predator-prey models. J. Appl. Probab. (1988) 25:612–616Crossref, Google Scholar
• Sennott L. I.Stochastic Dynamic Programming and the Control of Queueing System (1999) (Wiley, New York) Google Scholar
• Yushkevich A. A., Feinberg E. A. On homogeneous Markov model with continuous time and finite or countable state space. Theory Probab. Appl. (1979) 24:156–161Crossref, Google Scholar