Autoregressive Process Proofs

Basic Properties

Property 1: The mean of the y_i in a stationary AR(p) process is

Proof: Since the process is stationary, for any k, E[y_i] = E[y_i-k], a value which we will denote μ. Since E[ε_i] = 0, E[φ₀] = φ₀ and

it follows that

Solving for μ yields the desired result.

Property 2: The variance of the y_i in a stationary AR(1) process is

Proof: Since the y_i and ε_i are independent, by basic properties of variance, it follows that

Since the process is stationary, var(y_i) = var(y_i-1), and so

Solving for var(y_i) yields the desired result.

Property 3: The lag h autocorrelation in a stationary AR(1) process is

Proof: First note that for any constant a, cov(a+x, a+y) = cov(x,y). Thus, cov(y_i,y_j) has the same value even if we assume that φ₀ = 0, and similarly for var(y_i) = cov(y_i,y_i). Thus, it suffices to prove the property when φ₀ = 0. In this case, by Property 1, μ = 0, and so cov(y_i,y_j) = E[y_iy_j].

Thus

since by the stationary property, E[y_i-1,y_i-k] = γ_i-1. Now, by induction on k, it is easy to see that

HenceYule-Walker equations

Property 4 : For any stationary AR(p) process, we can calculate the autocovariance at lag k > 0 by

Similarly the autocorrelation at lag k > 0 can be calculated as

Proof: As usual, we can assume that the mean is zero (and so φ₀ = 0), Thus, we are dealing with the process

Thus

The second form of the property follows by dividing both sides of the equation by γ₀.

Proof of Property 7

Property 7: The variance of the y_i in a stationary AR(2) process is

Proof:

By the stationary property

and so

But by Property 4, 

and so

The result follows by expanding the terms in the denominator and seeing that these are equal to the expansion of the denominator in the statement of the property.

Reference

Alonso, A. M., Garcia-Martos, C. (2012) Time series analysis: autoregressive, MA and ARMA processes
https://www.academia.edu/35659911/Time_Series_Analysis_Autoregressive_MA_and_ARMA_processes