Easy Affine Markov Decision Processes: Properties and Applications
Abstract
This tutorial introduces a class of decomposable affine Markov decision processes (MDPs) that have continuous multidimensional endogenous states and actions, and an exogenous state that follows an exogenous Markov chain. We show that, unlike most MDPs with continuous state and actions, decomposable affine MDPs are free of the curse of dimensionality and can be solved easily and exactly. These nice properties are attributed to its affine dynamics and affine single-period rewards, its decomposable action space, and the polyhedral features of the decomposed action space. Exploiting its structure, we demonstrate that a decomposable affine MDP with a finite-horizon criterion has a value function that is affine in the endogenous state and has an extremal optimal policy; the value function and the extremal optimal policy are determined by the solution of a set of auxiliary equations. At the end of the tutorial, we illustrate the potential applicability of decomposable affine MDPs using the examples of fishery management and dynamic capacity portfolio management.
Your Access Options
-
Login Options
Purchase Options
Save for laterOther Options
Token AccessClaim access using a tokenRestore guest accessApplies for purchases made as a guest
