Dynamic Programming and Systems of Uncertain Duration

This paper deals with the application of dynamic programming to systems whose duration is uncertain, and which duration may or may not be a function of the decisionmaker's policy. It extends some of Bellman's work in that the transformed states at successive stages may not have the properties demanded by Bellman; e.g., the state norm need not diminish at all, and can, in fact, increase. The paper deals primarily with computational considerations since, in general, analytic solutions to functional equations are not forthcoming. Both discrete and continuous formulations are considered, the appropriate one to choose depending on the number of states one wishes to cater for, the continuous formulation having certain advantages which the discrete case does not have when a very large number of states are to be differentiated.