Inspection Procedures for Markov Chains

Consider a production process where the categories of successive units form a Markov chain with known transition matrix. Whenever a unit is produced, we may, at a fixed cost, inspect it. If its value is less than maximal, we may replace it with a perfect (maximal value) unit. We obtain optimal inspection procedures (procedures for which the value of the outgoing units is high and which keep the cost of inspection down).