Limiting Markov Chain Distributions

Basic Concepts

A distribution π of a Markov chain, as defined in Markov Chain Distributions, is a limiting distribution if for all states i, j

We get the same limit no matter what the starting value is. Thus

Equivalently, we say that a distribution π* is a limiting (aka equilibrium) distribution if

where P is the transition matrix, no matter what the initial distribution π is. Also

This means that for any set of initial states

is a square matrix of identical rows, where each row is π*

Key Properties

Property 1: A limiting distribution must be a stationary distribution.

Proof:

Property 2: If a finite Markov chain is irreducible and aperiodic, then a unique stationary distribution π exists where π_i = 1/n_i, and this distribution is a limiting distribution.

If the finite Markov chain has multiple classes, then any closed class has a unique stationary distribution π for that class, which is also a limiting distribution for that class, and so for any state in the class

Any transient class doesn’t have a stationary (or limiting) distribution, and for any state j in that class

Property 3: If a countable, infinite Markov chain is irreducible and aperiodic, then a unique stationary distribution π exists where π_i = 1/n_i, and this distribution is a limiting distribution, provided the Markov chain is positive recurrent. If the Markov chain is transient or null recurrent, then by Property 3 of Positive and Null Recurrence, no stationary (or limiting) distribution exists, and for any state j

If countable, infinite Markov chain has multiple classes, then any positive recurrent class has a unique stationary distribution π for that class, which is also a limiting distribution for that class, and so for any state in the class

Any transient or null recurrent class doesn’t have a stationary (or limiting) distribution, and for any state j in that class

Examples

We start by looking at the three examples from Positive and Null Recurrence.

Example 1: Find the limiting distribution for the Markov chain with transition matrix P in range B2:D4 of Figure 1.

We calculate Pⁿ for a sufficiently large value of n. For this example, we use the formula =MPRODUCT(B2:D4,20), as shown on the right side of Figure 1. We see that each row of the output is the stationary/limiting distribution of the Markov chain, (.076923, .230769, .692308). The reciprocals of these value yield n₁ = 13, n₂ = 4.3333, n₃ = 1.4444.

Figure 1 – Irreducible Markov chain

This is the same result that we obtained for Example 1 of Positive and Null Recurrence.

Example 2: Find the limiting distribution for the Markov chain with transition matrix P in range B2:E5 of Figure 2.

Using the formula =MPRODUCT(B2:E5,25), we obtain the results shown in range L2:O5 of Figure 2. (.33333, .66667) is the stationary/limiting distribution for {1, 2} and (.53333, 466667) is the stationary/limiting distribution for {3, 4}.  The reciprocals of these values produce the values for n₁, n₂, n₃, and n_4, as shown in range L7:O7.

Figure 2 – Two closed classes

Example 3: Find the limiting distribution for the Markov chain with transition matrix P in range B2:E5 of Figure 3.

Using the formula =MPRODUCT(B2:E5,25) in range G2:J5 of Figure 3, we obtain the stationary/limiting distribution for the closed class {3, 4}. 

Figure 3 – One closed class and one transient class

Using the formula =MPRODUCT(B2:E5, 200) in range G7:J10. we see evidence that P(x_n = 1) → 0 and P(x_n = 2) → 0 as n → ∞.

Periodic Example

Example 4: Find the limiting distribution for the Markov chain with transition matrix P in range B2:D4 of Figure 1.

We first note that this chain is periodic, where d₁ = d₂ = d₃ = 3 (see Periodocity). Using the eigVECT(B2:D2) formula in range J1:L7, we see that (1/3, 1/3, 1/3) is a stationary distribution. However, no limiting distribution exists since the limit of the Pⁿ doesn’t exist, oscilating between the values P³ⁿ⁺¹, P³ⁿ⁺², and P³ⁿ, as shown in ranges B2:E4, F2:H4, and F6:H8.

Figure 4 – Periodic chain

Ergodic Theorem

Property 4: Let

If the Markov chain is irreducible and positive recurrent, then for any initial distribution, there is a unique stationary distribution π such that

This distribution also has the property that π_i = 1/n_i.

If the Markov chain is irreducible and transient or null recurrent, then

Aperiodicity is not required for this property.

Example 5: For the Markov chain of Example 1, we see that in the long run, 7.69% of the time the Markov chain is in state 1, 23.08% of the time it is in state 2, and 69.23% of the time it is in state 3.

Example 6: For the Markov chain of Example 4, in the long term, 1/3 of the time the Markov chain is in each of the three states. Note that the theorem applies even though this Markov chain is periodic.

Examples Workbook

Click here to download the Excel workbook with the examples described on this webpage.

Links

↑ Markov chains

References

Aldridge, M. (2021) Long-term behaviour of Markov chains. Introduction to Markov Processes
https://mpaldridge.github.io/math2750/S11-long-term-chains.html

Norris, J. (2004) Discrete-time Markov chains. Cambridge University Press
https://www.statslab.cam.ac.uk/~jrn10//Markov/

Pishro-Nik, H. (2021) Stationary and limiting distributions
https://www.probabilitycourse.com/chapter11/11_2_6_stationary_and_limiting_distributions.php