Multivariate Regression Hypothesis Testing

Introduction

We start by showing how to test the hypothesis that none of the independent variables in a multivariate regression model predicts any of the dependent variables.

Basic Concepts

The regression coefficients takes the form

Here β₀ is a 1 × m matrix containing the intercepts and β₍₁₎ is a k × m matrix containing the coefficients associated with the independent variables.

We test the hypotheses

H₀: β₍₁₎ = O

H₁: β₍₁₎ ≠ O, i.e. β_jp ≠ 0 for some j = 1 ,…, k and p = 1, …, m

Now, let

where Y_j is the jth column of the data in Y. Then the sum of squares without taking the intercept into account is

But

where

Note that E = SSE.

As for MANOVA, we can use the following four tests to determine whether null hypothesis β₍₁₎ = O. For each of the tests we use the following parameters:

p = m, h = k, and e = n–k–1

In the following |A| is the determinant of the square matrix A, and λ₁ ≥ λ₂ ≥ …. ≥ λ_r are the eigenvalues of E^-1H (or HE^-1) with r = min(k,m).

Wilk’s Lambda Test

Define

In particular, if the null hypothesis is true, then

where

This is the same test used for one-way MANOVA where in that case p = k, h = m–1, and e = n–m.

Hotelling-Lawley Trace Test

Define

In particular, if the null hypothesis is true, then

where

Pillai-Bartlett Trace Test

Define

If the null hypothesis is true, then

where s is as described above and

 

Roy Largest Root Test

Define

where λ₁ is the largest eigenvalue of HE^-1.

There isn’t an F-test version of Roy’s test. Instead, you can use a table of critical values for alpha = .05 based on the parameters s, t, u from the Hotelling-Lawley Trace Test.

You reject the null hypothesis when Θ > Θ_crit.

Related Topics

• Significance testing example
• Testing a subset of coefficients
• Real Statistics support and examples

References

Johnson, R. A., Wichern, D. W. (2007) Applied multivariate statistical analysis. 6th Ed. Pearson
https://mathematics.foi.hr/Applied%20Multivariate%20Statistical%20Analysis%20by%20Johnson%20and%20Wichern.pdf

Rencher, A.C., Christensen, W. F. (2012) Methods of multivariate analysis (3^nd Ed). Wiley

Junker, B. (2005) Multivariate regression, MANOVA
https://www.stat.cmu.edu/~brian/722-2005/week04/07-reg-topics2.pdf