Bayesian Hypothesis Testing

Objective

We show how to perform hypothesis testing for normally distributed data using the Bayesian approach described at Bayesian Hypothesis Testing.

On this webpage, we focus on the following one-sided tests:

• One sample test with
  • known variance
  • unknown variance
• Two sample test
  • equal fixed variance
  • equal unknown variance
  • unequal unknown variances

One-sample test with known variance

Suppose we have a sample X = x₁, …, x_n that comes from a normally distributed population with known fixed variance; i.e. x_i ~ N(μ, σ²) for all i. We test the following null and alternative hypotheses.

H₀: μ < 0

H₁: μ ≥ 0

We assume the Jeffreys’ prior f(μ) ∝ 1 (see Non-informative Priors). Thus, the posterior is proportional to the likelihood function, and so

The posterior probability of the null-hypothesis is therefore

where Φ is the cdf for the standard normal distribution and

If your decision criterion for rejecting H₀ in favor of H₁ is that P(H₀|X) < α, then you can use the frequentist one-sided z-test, namely reject the null hypothesis if –z < z_α or equivalently +z > z_1-α

One-sample test with unknown variance

We repeat the above one-sided test where the variance is unknown. This time we use the Jeffreys’ prior

f(μ, σ) = σ^-3

(see Non-informative Priors). The resulting posterior for μ is

where t_n is the non-standardized t-distribution.

This is the same as the usual frequentist t-test except that s² is defined with division by n-1 instead of n, and the degrees of freedom is n-1 instead of n.

Click here for a proof of the above assertion.

Two-sample test with equal known variance

Suppose we have a samples X = x₁, …, x_m and Y = y₁, …, y_n that comes from a normally distributed population with known fixed variance; i.e. x_i ~ N(μ_x, σ²) and y_i ~ N(μ_y, σ²) for all i. We test the following null and alternative hypotheses where δ = μ_y – μ_x.

H₀: μ_x < μ_y (or δ > 0)

H₁: μ_x ≥ μ_y (or δ ≤ 0)

We use the Jeffreys’ prior f(μ_x, μ_y) = 1. It can be shown that the posterior distribution is

Once again, this takes the form of the frequentist test with a cleaner interpretation.

Two-sample test with equal unknown variances

This time we use the Jeffreys’ prior f(μ_x, δ, σ²) ∝ (σ²)^-2. It turns out that the posterior (calculated by integrating over μ_x and σ²) is 

where the pooled variance is

with

Two-sample test with unequal unknown variances

Since neither group shares any parameters, we can use the one sample approach to obtain

Under construction

References

Reich, B. J., Ghosh, S. K. (2019) Bayesian statistics methods. CRC Press

Lee, P. M. (2012) Bayesian statistics an introduction. 4^th Ed. Wiley
https://www.wiley.com/en-us/Bayesian+Statistics%3A+An+Introduction%2C+4th+Edition-p-9781118332573

Jordan, M. (2010) Bayesian modeling and inference. Lecture 1. Course notes
https://people.eecs.berkeley.edu/~jordan/courses/260-spring10/lectures/lecture1.pdf

Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., Rubin, D. B. (2014) Bayesian data analysis, 3^rd Ed. CRC Press
https://statisticalsupportandresearch.files.wordpress.com/2017/11/bayesian_data_analysis.pdf