Optimal Plans for Dynamic Programming Problems

It is proved that there exist stationary optimal plans for discounted dynamic programming problems, and that there exist semi-Markov ϵ-optimal plans for positive dynamic programming problems. The actions are required to be taken in a variable action set F(s), and the reward function r(s, a) is a Borel measurable function of (s, a) and an u.s.c. function of a. Our results are related to recent work by Furukawa, Maitra, and Schal. The key tool is a generalization of a selection theorem of Dubins and Savage.