Technical Note—Cyclic Dynamic Programming: A Procedure for Problems with Fixed Delay

A dynamic decision problem in which the effect of control action is either delayed for a number of time periods or has an effect that lasts for a fixed period leads, in the conventional formulation, to a high-dimensional dynamic-programming problem. This paper develops a method for exploiting the inherent structure of such problems that leads to low-dimensional “cyclic dynamic programming” problems that are easier to solve than the problem that is obtained by the conventional approach.