Average Optimality in Dynamic Programming with General State Space

A Markovian decision model with general state space, compact action space, and the average cost as criterion is considered. The existence of an optimal policy is shown via an optimality inequality in terms of the minimal average cost g and a relative value function w. The existence of some w is usually shown via relative compactness in a space of real-valued functions on the state space. Here it shall be shown that one can instead do with pointwise relative compactness in the set of real numbers if one makes use of a (generalized) lower limit of functions. An application to an inventory model is given.