Solution Representations for Poisson’s Equation, Martingale Structure, and the Markov Chain Central Limit Theorem

References

• Asmussen S (2008) Applied Probability and Queues (Springer Science, New York).Google Scholar
• Awad HP, Glynn PW (2007) On the theoretical comparison of low-bias steady-state estimators. ACM Trans. Model. Comput. Simul. 17(1):4.Google Scholar
• Barlow RE, Proschan F (1975) Statistical Theory of Reliability and Life Testing: Probability Models (Holt, Rinehart and Winston, New York).Google Scholar
• Bhattacharya RN (1982) On the functional central limit theorem and the law of the iterated logarithm for Markov processes. Zeitschrift Wahrscheinlichkeitstheorie Verwandte Gebiete 60(2):185–201.Google Scholar
• Bhulai S, Spieksma FM (2003) On the uniqueness of solutions to the Poisson equations for average cost Markov chains with unbounded cost functions. Math. Methods Oper. Res. 58(2):221–236.Google Scholar
• Chung KL (1967) Markov Chains: With Stationary Transition Probabilities, 2nd ed. (Springer, Berlin).Google Scholar
• Chung KL (1974) A Course in Probability Theory (Academic Press, New York).Google Scholar
• Daley DJ (1968) The serial correlation coefficients of waiting times in a stationary single server queue. J. Australian Math. Soc. 8(4):683–699.Google Scholar
• Derman C, Veinott AF (1967) A solution to a countable system of equations arising in Markovian decision processes. Ann. Math. Statist. 38(2):582–584.Google Scholar
• Frenk JB (1982) The behavior of the renewal sequence in case the tail of the waiting-time distribution is regularly varying with index −1. Adv. Appl. Probab. 14(4):870–884.Google Scholar
• Glynn PW (1994) Poisson’s equation for the recurrent M/G/1 queue. Adv. Appl. Probab. 26(4):1044–1062.Google Scholar
• Glynn PW, Meyn SP (1996) A Liapounov bound for solutions of the Poisson equation. Ann. Probab. 24(2):916–931.Google Scholar
• Glynn PW, Ormoneit D (2002) Hoeffding’s inequality for uniformly ergodic Markov chains. Statist. Probab. Lett. 56(2):143–146.Google Scholar
• Glynn PW, Whitt W (1993) Limit theorems for cumulative processes. Stochastic Processes Their Appl. 47(2):299–314.Google Scholar
• Glynn PW, Whitt W (2002) Necessary conditions in limit theorems for cumulative processes. Stochastic Processes Their Appl. 98(2):199–209.Google Scholar
• Glynn PW, Jacobovic R, Mandjes M (2023) Moments of polynomial functionals in Levy-driven queues with secondary jumps. Preprint, submitted October 17, https://arxiv.org/abs/2310.11137.Google Scholar
• Gordin MI, Lifšic BA (1978) The central limit theorem for stationary Markov processes. Doklady Akademii Nauk 239(4):766–767.Google Scholar
• Hall P, Heyde CC (2014) Martingale Limit Theory and its Application (Academic Press, New York).Google Scholar
• Hernández-Lerma O, Lasserre JB (1999) Ergodicity and Poisson’s equation. Karatzas I, Yor M, eds. Further Topics on Discrete-Time Markov Control Processes (Springer, New York), 1–38.Google Scholar
• Jones GL (2004) On the Markov chain central limit theorem. Probab. Surveys 1:299–320.Google Scholar
• Kiefer J, Wolfowitz J (1956) On the characteristics of the general queueing process, with applications to random walk. Ann. Math. Statist. 27(1):147–161.Google Scholar
• Kurtz TG (1981) The central limit theorem for Markov chains. Ann. Probab. 9(4):557–560.Google Scholar
• Maigret N (1978) Théorème de limite centrale fonctionnel pour une chaîne de Markov récurrente au sens de Harris et positive. Ann. Inst. Henri Poincare B: Probab. Statist. 14(4):425–440.Google Scholar
• Makowski AM, Shwartz A (2002) The Poisson equation for countable Markov chains: Probabilistic methods and interpretations. Feinberg EA, Shwartz A, eds. Handbook of Markov Decision Processes (Springer, Boston), 269–303.Google Scholar
• Meyn SP, Tweedie RL (2009) Markov Chains and Stochastic Stability, 2nd ed. (Cambridge University Press, Cambridge, UK).Google Scholar
• Nummelin E (1991) On the Poisson equation in the potential theory of a single kernel. Math. Scand. 68(1):59–82.Google Scholar
• Revuz D (1975) Markov Chains (North-Holland/American Elsevier, New York).Google Scholar
• Rhee CH, Glynn PW (2022) Lyapunov conditions for differentiability of Markov chain expectations. Math. Oper. Res. 48(4):2019–2042.Google Scholar
• Ross SM (2014) Introduction to Stochastic Dynamic Programming (Academic Press, New York).Google Scholar
• Schaufele RA (1967) A potential theoretic proof of a theorem of Derman and Veinott. Ann. Math. Statist. 38(2):585–587.Google Scholar
• Veretennikov A (2017) Ergodic Markov processes and Poisson equations. Panov V, ed. Modern Problems of Stochastic Analysis and Statistics: Selected Contributions in Honor of Valentin Konakov (Springer, Cham, Switzerland), 457–511.Google Scholar