Stochastic Optimization by Simulation: Numerical Experiments with the M/M/1 Queue in Steady-State

This paper gives numerical illustrations of the behavior of stochastic approximation, combined with different derivative estimation techniques, to optimize a steady-state system. It is a companion paper to L'Ecuyer and Glynn (1993), which gives convergence proofs for most of the variants experimented here. The numerical experiments are made with a simple M/M/1 queue, which while simple, serves to illustrate the basic convergence properties and Possible pitfalls of the various techniques.