GLS Method for Autocorrelation

Objective

Even when autocorrelation is present the OLS coefficients are unbiased, but they are not necessarily the estimates of the population coefficients that have the smallest variance. We now demonstrate the generalized least squares (GLS) method for estimating the regression coefficients with the smallest variance.

GLS Approach

Suppose that the population linear regression model is

and so for each observation i

Now suppose that all the linear regression assumptions hold, except that there is autocorrelation, i.e. E[ε_iε_i+h] ≠ 0 where h ≠ 0. Let’s assume, in particular, that we have first-order autocorrelation, and so for all i, we can express ε_i by

ε_i = ρε_i-1 + δ_i

where ρ is the first-order autocorrelation coefficient, i.e. the correlation coefficient between ε₁, ε₂, …, ε_n-1  and ε₂, ε₃, …, ε_n and the u_i is an error term that satisfies the standard OLS assumptions, namely E[δ_i] = 0, var(δ_i) = σ_δ, a constant, and cov(δ_i,δ_j) = 0 for all i ≠ j. Note that since ρ is a correlation coefficient, it follows that -1 ≤ ρ ≤ 1.

We will assume further that -1 < ρ < 1, which means that the sequence ε₁, ε₂, …, ε_n has some desirable properties (namely that when viewed as a time series it is stationary).

Differencing

Now

and so

Multiplying both sides of the second equation by ρ and subtracting it from the first equation yields

Note that ε_i – ρε_i-1 = δ_i, and if we set

for all j > 0, then this equation can be expressed as the generalized difference equation:

This equation satisfies all the OLS assumptions and so an estimate of the parameters β₀^′,  β₁, …, β_k can be found using the standard OLS approach provided we know the value of ρ.

Prais-Winsten Transformation

Note that we lose one sample element when we utilize this differencing approach since y₁ and the x_1j have no predecessors. For large samples, this is not a problem, but it can be a problem with small samples. We can use the Prais-Winsten transformation to obtain a first observation, namely

Second-order Autocorrelation

The same approach can be used with p-order autocorrelation. E.g. for second-order autocorrelation, we have

Thus

We can multiply the second equation by ρ₁ and the third equation by ρ₂, and then subtract both of these from the first equation to once again get the equation

But this time

As before, this equation satisfies all the OLS assumptions and so an estimate of the parameters β₀^′,  β₁, …, β_k can be found using the standard OLS approach provided we know the value of ρ₁ and ρ₂. Except for the intercept, the resulting coefficients are coefficients for the original regression model. The intercept coefficient of the original model is simply β₀^′/(1 – ρ₁ – ρ₂). The same approach is used for the standard errors of the original regression model.

References

Wooldridge, J. M. (2013) Introductory econometrics, a modern approach (5th Ed). Cengage Learning

Wikipedia (2018) Generalized least squares
https://en.wikipedia.org/wiki/Generalized_least_squares