Gibbs Sampler Normal Distribution

Gibbs Sampler Algorithm

As described in Gibbs Sampler Two Sample Binomial, suppose we want a sample from the joint distribution f(θ, φ), but we don’t have access to this distribution directly, although we do have access to the distributions for θ|φ and φ|θ. In this case, we can use the following iteration method, called Gibbs Sampler.

  • Start with a guess θ0 (which is any permissible value for θ)
  • For each i > 0, get sample values θi ∼ f(θ|φi-1) and φi ∼ f(φ|θi)

Then for sufficiently large i, (θi, φi) ∼ f(θ, φ).

Gibbs Sampler for bivariate normal distribution

Example 1: Create a sample of size 2,000 from a bivariate normal distribution with μ1 = 2, μ2 = 1, σ1 = .5, σ2 = 1.5 and ρ = .6 using Gibbs Sampler.

We create a sample of 2,100 points (xi, yi) where we throw away 100 of the initial values. We start with an initial value for x0 of zero, and then at each iteration, we look for random values such that

x_i conditional probability

y_i conditional probability

The result is shown in Figure 1 where only the first five sample pairs (xi, yi) are displayed and where range B2:B6 shows the target parameters of the bivariate normal distribution.

Gibbs sampler bivariate normal

Figure 1 – Gibbs Sample for bivariate normal distribution

Representative Formulas

Representative formulas from Figure 1 are shown in Figure 2. Range E2:E6 shows the calculated parameter values based on the sample generated in range B109:C2108. We see that these values are fairly close to the values in range B2:B6, as expected.

Representative formulas

Figure 2 – Representative formulas from Figure 1

References

Peng, R. D. (2021) Advanced statistical computing

https://bookdown.org/rdpeng/advstatcomp/

https://bookdown.org/rdpeng/advstatcomp/gibbs-sampler.html#example-bivariate-normal-distribution

https://bookdown.org/rdpeng/advstatcomp/gibbs-sampler.html#example-normal-likelihood

Leave a Comment