Optimal Replacement Rules when Changes of State are Semi-Markovian

This paper investigates the use of a discrete-time semi-Markov process to model a system that deteriorates in usage. Replacement rules that are (1) state-dependent, (2) state-age-dependent, and (3) age-dependent are proposed. The system operating costs and replacement costs are functions of the underlying states. The optimization criterion is the expected average cost per unit time. Under the first two replacement rules, the paper generates semi-Markov decision processes so that optimal policies can be obtained by the policy-iteration method. Sufficient conditions for the existence of an optimal control-limit state-dependent replacement rule are derived. For the age-dependent policy, the objective function is obtained so that the minimization can be carried out over the integers. An illustrative example is given at the end.