Higher-Order and Average Reward Myopic-Affine Dynamic Models

The dynamics and rewards in a higher-order model depend directly on states and actions in earlier periods. Here, the expected values of the single-period reward and of the state in the next period are affine functions of the states in the current and earlier periods. It is shown that computations in higher-order infinite-horizon models with myopic solutions are not essentially harder than in the corresponding first-order models. These results are obtained for discounted and average-reward criteria, and similar conclusions are indicated for sequential games. The results are applied to an aquaculture harvesting model and an advertising model with delayed responses.