Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach

References

• Altman E.Constrained Markov Decision Processes (1999) (Chapman & Hall/CRC, Boca Raton, FL) Google Scholar
• Bertsekas D. P.Dynamic Programming and Optimal Control, Volume II (1995) (Athena Scientific, Belmont, MA) Google Scholar
• Bertsekas D. P., Shreve S. E.Stochastic Optimal Control: The Discrete-Time Case (1978) (Academic Press, New York) . (Republished by Athena Scientific, 1997.)Google Scholar
• Borkar V. S., Feinberg E. A., Shwartz A. Convex analytic methods in Markov decision processes. Handbook of Markov Decision Processes: Methods and Applications (2002) (Kluwer, Boston, MA) 347–375Crossref, Google Scholar
• Cassandras C. G.Discrete Event Systems (1993) (IRWIN, Boston, MA) Google Scholar
• Derman C., Strauch R. E. A note on memoryless rules for controlling sequential control processes. Ann. Math. Statist. (1966) 37:276–278Crossref, Google Scholar
• Dynkin E. B., Yushkevich A. A.Controlled Markov Processes (1979) (Springer, Berlin) Crossref, Google Scholar
• Feinberg E. A. A generalization of “expectation equals reciprocal of intensity” to non-stationary exponential distributions. J. Appl. Probab. (1994) 31:262–267Crossref, Google Scholar
• Feinberg E. A., How Z., Filar J. A., Chen A. Constrained discounted semi-Markov decision processes. Markov Processes and Controlled Markov Chains (2002) (Kluwer, Dordrecht, The Netherlands) 233–244Crossref, Google Scholar
• Feinberg E. A. Constrained finite continuous-time Markov decision processes with average rewards. Proc. 2002 IEEE Conference on Decisions and Control (2002a) December 2002Las Vegas(IEEE Control Systems Society, Piscataway, NJ) 3805–3810Crossref, Google Scholar
• Feinberg E. A., Piunovskiy A. B. Nonatomic total reward Markov decision processes with multiple criteria. J. Math. Anal. Appl. (2002) 273:93–111Crossref, Google Scholar
• Feinberg E. A., Shwartz A. Constrained Markov decision models with weighted discounted rewards. Math. Oper. Res. (1995) 20:302–320Link, Google Scholar
• Feinberg E. A., Shwartz A. Constrained discounted dynamic programming. Math. Oper. Res. (1996) 21:922–945Link, Google Scholar
• Gonzáles-Hernández J., Hernández-Lerma O. Envelopes of sets of measures, tightness, and Markov control processes. Appl. Math. Optim. (1999) 40:377–392Crossref, Google Scholar
• Hernández-Lerma O., Gonzáles-Hernández J. Constrained Markov control processes in Borel spaces. Math. Meth. Oper. Res. (2000) 52:271–285Crossref, Google Scholar
• Heyman D. P., Sobel M. J.Stochastic Models in Operations Research, Volume II: Stochastic Optimization (1984) (McGraw-Hill, New York) Google Scholar
• Hordijk A., van der Duyn Schouten F. A. Discretization procedures for continuous time decision processes. Trans. Eighth Prague Conf. on Inform. Theory, Statistical Decision Functions, Random Processes, 1978, Volume C (1979) (Academia, Prague, Czechoslovakia) 143–154Google Scholar
• Jacod J. Multivariate point processes: Predictable projections, Radon-Nikodym derivatives, representation of martingales. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete (1975) 31:235–253Crossref, Google Scholar
• Kakumanu P. Continuously discounted Markov decision models with countable state and action space. Ann. Math. Statist. (1971) 42:919–926Crossref, Google Scholar
• Kakumanu P. Relation between continuous and discrete time Markovian decision problems. Naval Res. Logist. Quart. (1977) 24:431–439Crossref, Google Scholar
• Kallianpur G.Stochastic Filtering Theory (1980) (Springer, New York) Crossref, Google Scholar
• Kitaev Yu. M. Semi-Markov and jump Markov controlled models: Average cost criterion. SIAM Theory Probab. Appl. (1985) 30:272–288Crossref, Google Scholar
• Kitaev Yu. M., Rykov V. V.Controlled Queueing Systems (1995) (CRC Press, New York) Google Scholar
• Lippman S. A. Applying a new device in the optimization of exponential service systems. Oper. Res. (1975) 23:687–710Link, Google Scholar
• Miller B. L. Finite state continuous time Markov decision processes with a finite planning horizon. SIAM J. Control (1968) 6:266–280Crossref, Google Scholar
• Miller B. L. Finite state continuous time Markov decision processes with an infinite planning horizon. J. Math. Anal. Appl. (1968a) 22:552–569Crossref, Google Scholar
• Neveu J.Mathematical Foundations of the Calculus of Probability (1965) (Holden-Day, San Francisco, CA) Google Scholar
• Parthasarathy K. R.Probability Measures on Metric Spaces (1967) (Academic Press, New York) Crossref, Google Scholar
• Pinedo M.Scheduling: Theory, Algorithms, and Systems (1995) (Prentice Hall, Englewood Cliffs, NJ) Google Scholar
• Piunovskiy A. B.Optimal Control of Random Sequences in Problems with Constraints (1997) (Kluwer, Boston, MA) Crossref, Google Scholar
• Piunovskiy A. B. A controlled jump discounted model with constraints. SIAM Theory Probab. Appl. (1997a) 42:51–72Crossref, Google Scholar
• Puterman M. L.Markov Decision Processes (1994) (John Wiley, New York) Crossref, Google Scholar
• Ross S. M.Introduction to Stochastic Dynamic Programming (1983) (Academic Press, New York) Google Scholar
• Sennott L. I.Stochastic Dynamic Programming and the Control of Queueing Systems (1999) (Wiley, New York) Google Scholar
• Serfozo R. F. An equivalence between continuous and discrete time Markov decision processes. Oper. Res. (1979) 27:616–620Link, Google Scholar
• Yushkevich A. A. Controlled Markov models with countable state space and continuous time. SIAM Theory Probab. Appl. (1977) 22:215–235Crossref, Google Scholar
• Yushkevich A. A. On reducing a jump controllable Markov model to a model with discrete time. SIAM Theory Probab. Appl. (1980) 25:58–69Crossref, Google Scholar
• Yushkevich A. A. Controlled jump Markov models. SIAM Theory Probab. Appl. (1980a) 25:244–266Crossref, Google Scholar
• Yushkevich A. A., Feinberg E. A. On homogeneous Markov models with continuous time and finite or countable state space. SIAM Theory Probab. Appl. (1979) 26:156–161Crossref, Google Scholar