On-Line Optimization of Simulated Markovian Processes

Let {Z_n} be a Markovian process, the transition of which depends on a control parameter x. Let μ_x be its invariant law. It is shown that the solution of the optimization problem

F(x) := ∫ H(z, x)dμ_x(z) = min!, x ∈ S

can be found with a recursive estimation procedure of the stochastic approximation-type. The method consists in finding a stochastic quasigradient of F(x) and in adapting the parameter x in the direction of descent. An a.s. convergence is proved and a practical example is given.