A Method for Approximate Solutions to Stochastic Dynamic Programming Problems Using Expectations

This note describes and illustrates a computational technique to obtain approximate solutions to stochastic dynamic programming problems. The technique is to replace probability distributions by their corresponding expectations, and to use the values of the states in the corresponding deterministic system under its optimal policy to determine an approximate policy in the stochastic system through a single application of Howard's policy improvement operation. Two examples are given.