Limit Equilibrium Payoffs in Stochastic Games

We study the limit of equilibrium payoffs, as the discount factor goes to one, in non-zero-sum stochastic games. We first show that the set of stationary equilibrium payoffs always converges. We then provide two-player examples in which the whole set of equilibrium payoffs diverges. The construction is robust to perturbations of the payoffs and to the introduction of normal-form correlation.