Optimal Replacement and Maintenance Under Markovian Deterioration with Probability Bounds on Failure

A system is observed periodically and classified into one of a finite number of states. On the basis of the observations certain maintenance or replacement decisions are made. Transitions probabilities from state to state are determined by the decisions. The problem considered is that of finding the decision rule which maximizes the expected length of time between replacements subject to the side conditions that the probabilities of replacement through certain undesirable states are bounded by prescribed numbers.