A Convex Analytic Framework for Ergodic Control of Semi-Markov Processes

The ergodic control problem for semi-Markov processes is reformulated as an optimization problem over the set of suitably defined ‘ergodic occupation measures.’ This set is shown to be closed and convex, with its extreme points corresponding to stationary strategies. This leads to the existence of optimal stationary strategies under additional hypotheses. A pathwise analysis of the joint empirical occupation measures of the state and control processes shows that this optimality is in the strong (i.e., almost sure) sense.