Markov Decision Processes with Exogenous Variables

References

• Adelman D, Mancini AJ (2016) Optimality of quasi-open-loop policies for discounted semi-Markov decision processes. Math. Oper. Res. 41(4):1222–1247.Link, Google Scholar
• Aguirregabiria V, Magesan A (2013) Euler equations for the estimation of dynamic discrete choice structural models. Adv. Econometrics 31(46056):1–42.Google Scholar
• Aguirregabiria V, Magesan A (2018) Solution and estimation of dynamic discrete choice structural models using Euler equations. Working paper, University of Toronto, Toronto.Google Scholar
• Aguirregabiria V, Mira P (2010) Dynamic discrete choice structural models: A survey. J. Econometrics 156(1):38–67.Crossref, Google Scholar
• Bray RL (2019) Strong convergence and dynamic economic models. Quant. Econom. Forthcoming.Crossref, Google Scholar
• Chen C (2017) Slow focus: Belief evolution in the U.S. acid rain program. Working paper, John F. Kennedy School of Government, Cambridge, MA.Google Scholar
• Hajnal J (1957) Weak ergodicity in non-homogeneous Markov chains. Math. Proc. Cambridge Philos. Soc. 54(2):233–246.Crossref, Google Scholar
• Higle JL, Bean JC, Simth RL (1990) Deterministic equivalence in stochastic infinite horizon problems. Math. Oper. Res. 15(3):396–407.Link, Google Scholar
• Horn RA, Johnson CR (2013) Matrix Analysis (Cambridge University Press, New York).Google Scholar
• Morton TE (1971) On the asymptotic convergence rate of cost differences for Markovian decision processes. Oper. Res. 19(1):244–248.Link, Google Scholar
• Morton TE, Wecker WE (1977) Discounting, ergodicity and convergence for Markov decision processes. Management Sci. 23(8):890–900.Link, Google Scholar
• Paarsch HJ, Rust J (2009) Valuing programs with deterministic and stochastic cycles. J. Econom. Dynam. Control 33(3):614–623.Crossref, Google Scholar
• Puterman ML (2005) Markov Decision Processes: Discrete Stochastic Dynamic Programming (John Wiley & Sons, Hoboken, NJ).Google Scholar
• Seneta E (2006) Non-Negative Matrices and Markov Chains (Springer Science & Business Media, New York).Google Scholar
• Tauchen G (1986) Finite state Markov-chain approximations to univariate and vector autoregressions. Econom. Lett. 20(2):177–181.Crossref, Google Scholar