Regenerative Analysis and Steady State Distributions for Markov Chains

We apply regenerative theory to derive certain relations between steady state probabilities of a Markov chain. These relations are then used to develop a numerical algorithm to find these probabilities. The algorithm is a modification of the Gauss-Jordan method, in which all elements used in numerical computations are nonnegative; as a consequence, the algorithm is numerically stable.