Algorithms for Stochastic Games with Geometrical Interpretation

The paper presents a new approach, based on analysis and geometrical interpretation, to the solution of Markov stochastic games. The proposed algorithm, using iterations in policy space, turns out to be a Newton-Raphson type procedure. Several numerical examples are given, covering the terminating and non-terminating cases respectively and illustrating the advantages of the proposed algorithm compared with other known algorithms. Special attention is given to Howard's sequential decision problem with discrete and continuous policy spaces.