Decision Problems with Expected Utility Critera, I: Upper and Lower Convergent Utility

A countable stage, countable state, finite action decision problem is considered where the objective is the maximization of the expectation of an arbitrary utility function defined on the sequence of states. Basic concepts are formulated, generalizing the standard notions of the optimality equations, conserving and unimprovable strategies, and strategy and value iteration. Analogues of positive, negative and convergent dynamic programming are analyzed.