Measure-Valued Differentiation for Stationary Markov Chains

References

• Baccelli F., Hong D. Analytic expansions of (max,+) Lyapunov exponents. Ann. Appl. Probab. (2000) 10:779–827Crossref, Google Scholar
• Braker J. Max-algebra modelling and analysis of time-dependent transportation networks. Proc. 1st Eur. Control Conf. (1991) Grenoble, France:1831–1836Hermes, Paris, FranceGoogle Scholar
• Brémaud P. Maximal coupling and rare perturbation analysis. Queueing Systems: Theory Appl. (1992) 11:307–333Crossref, Google Scholar
• Brémaud P., Vázquez-Abad F. On the pathwise computation of derivatives with respect to the rate of a point process: The phantom RPA method. Queueing Systems: Theory Appl. (1992) 10:49–270Crossref, Google Scholar
• Borovkov A. A., Hordijk A. On normed ergodicity of Markov chains. (2000) . Technical report MI 2000-40 Leiden University, Leiden, The NetherlandsGoogle Scholar
• Borovkov A. A., Hordijk A. Characterization and sufficient conditions for normed ergodicity of Markov chains. Adv. Appl. Probab. (2004) 36:227–242Crossref, Google Scholar
• Cohen J. W.The Single Server Queue (1969) (North-Holland, Amsterdam, The Netherlands) Google Scholar
• Dekker R., Hordijk A. Average, sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards. Math. Oper. Res. (1988) 13:395–421Link, Google Scholar
• Dekker R., Hordijk A., Spieksma F. M. On the relation between recurrence and ergodicity properties in denumerable Markov decision chains. Math. Oper. Res. (1994) 19:539–559Link, Google Scholar
• Glynn P. Likelihood ratio gradient estimation for stochastic systems. Comm. ACM (1990) 33:75–84Crossref, Google Scholar
• Glynn P., L’Ecuyer P. Likelihood ratio gradient estimation for stochastic recursions. Adv. Appl. Probab. (1995) 27:1019–1053Crossref, Google Scholar
• Glynn P., L’Ecuyer P., Adès M., Nelson B., Kelton W., Clark G. Gradient estimation for ratios. Proc. 1991 Winter Simulation Conf. (1991) Winter Simulation ConferenceSan Diego, CA:986–993Google Scholar
• Heidergott B., Hordijk A., Weisshaupt H. Derivatives of Markov kernels and their Jordan decomposition. (2003) . EURANDOM Report 2003-001, EURANDOM, Eindhoven, The NetherlandsGoogle Scholar
• Heidergott B., Hordijk A., Weisshaupt H. Measure-valued differentaion for stationary Markov chains. (2002) . EURANDOM Report 2002-027, EURANDOM, Eindhoven, The NetherlandsGoogle Scholar
• Heidergott B., Cao X. A note on the relation between weak derivatives and perturbation realization. IEEE Trans. Automatic Control (2002) 47:1112–1115Google Scholar
• Heidergott B., Hordijk A. Taylor series expansions for stationary Markov chains. Adv. Appl. Probab. (2003) 35:1046–1070Crossref, Google Scholar
• Heidergott B., Hordijk A. Single-run gradient estimators via measure-valued differentiation. IEEE Trans. Automatic Control (2004) 49:1843–1846Crossref, Google Scholar
• Heidergott B., Vázquez-Abad F. Measure-valued differentiation for stochastic processes: The finite horizon case. (2000) . EURANDOM Report 2000-033, EURANDOM, Eindhoven, The NetherlandsGoogle Scholar
• Heidergott B., Vázquez-Abad F. Gradient estimation for a problem in public transportation: A comparison of SPA, SF and MVD. (2004) 241–246International Workshop on DES (WODES’04), Reims, FranceGoogle Scholar
• Heidergott B., de Vries R. Towards a control theory for transportation networks. Discrete Event Dynam. Systems (2001) 11:371–398Crossref, Google Scholar
• Hordijk A., Dekker R. Average, sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards. (1983) . Report No. 83-36, Institute of Applied Mathematics and Computing Science, Leiden University, Leiden, The NetherlandsGoogle Scholar
• Hordijk A., Spieksma F. M. On ergodicity and recurrence properties of a Markov chain with an application to an open Jackson network. Adv. Appl. Probab. (1992) 24:343–376Crossref, Google Scholar
• Hordijk A., Spieksma F. M., Kelly F. P. A new formula for the deviation matrix. Probability, Statistics and Optimization (1994) (Wiley, New York) . Chapter 36 inGoogle Scholar
• Hordijk A., Yushkevich A. A. Blackwell optimality in the class of all policies in Markov decision chains with a Borel state space and unbounded rewards. Math. Methods Oper. Res. (1999) 50:421–448Crossref, Google Scholar
• Kushner H., Vázquez-Abad F. Estimation of the derivative of a stationary measure with respect to a control parameter. J. Appl. Probab. (1992) 29:343–353Crossref, Google Scholar
• Meyn S. P., Tweedie R. L.Markov Chains and Stochastic Stability (1993) (Springer, London, UK) Crossref, Google Scholar
• Pflug G. Gradient estimates for the performance of Markov chains and discrete event processes. Ann. Oper. Res. (1992) 39:173–194Crossref, Google Scholar
• Pflug G.Optimisation of Stochastic Models (1996) (Kluwer Academic Publishers, Boston, MA) Crossref, Google Scholar
• Ross S. M.Applied Probability Models with Optimization Applications (1970) (Holden-Day, San Francisco, CA) Google Scholar
• Rubinstein R., Shapiro A.Discrete Event Systems: Sensitivity Analysis and Optimization by the Score Function Method (1993) (John Wiley and Sons, New York) Google Scholar
• Suri R., Cao X. The phantom customer and marked customer methods for optimization of closed queueing networks with blocking and general service times. Performance Evaluation Rev. (1983) August:243–256Google Scholar