Separable Markovian Decision Problems

The special structure of a class of Markovian decision problems is exploited to simplify the determination of optimum policies. For certain pairs consisting of a state i and decision k, the cost c^k_i separates (c^k_i = a_i + b_k), while the transition probabilities p^k_ij and transition time distributions F^k_ij are independent of i. Equivalence of a second Markovian decision problem which exploits this structure is demonstrated for the discounted and averaging cases. In addition, streamlined approaches are presented for dealing directly with the original problem, and a particular inventory model is further simplified.