On Stationary Strategies in Countable State Total Reward Markov Decision Processes

This paper deals with total reward Markov decision processes with countable state space. Various partial results from the literature are connected and extended in the following theorem. If in each state where the value is nonpositive a conserving action exists then there exists a stationary strategy f which is uniformly nearly optimal in the following sense: v(f) ≥ v* − ϵu*, where u* is the value of the problem if only the positive rewards are counted.

Further, the following result is established: if an optimal strategy exists then also an optimal stationary strategy exists.