Successive Approximations for Finite Horizon, Semi-Markov Decision Processes with Application to Asset Liquidation

This paper presents a simple successive approximation approach to the characterization of optimal policies for finite horizon, semi-Markov decision processes. Optimal policies are nonstationary, for in this setting they depend on both time and state. We illustrate this approach by analyzing the optimal liquidation of an asset; we also show that several aspects of the standard, discrete-time, infinite horizon optimal policy carry over to the continuous-time, finite horizon policy.