Jackknifing for Deming Regression

Jackknife Procedure

To calculate an estimate of the standard error of θ-hat which is an estimate of some parameter θ, we can use a nonparametric procedure called the Jackknife method. This uses the following procedure:

1. For each element x_i in the sample (actually the pair (x_i, y_i) for Deming regression), calculate θ_i-hat in the same way as θ-hat was calculated except that the element x_i is left out of the calculation.
2. Calculate 
3. Calculate the mean of these values, called the jackknifed estimate by using the following formula
4. The jackknifed estimate of the variance and standard error of is now given by the formulas

It is easy to see that the jackknifed estimate of the variance can also be calculated more simply as n – 1 times the deviation squared of the θ_i-hat values. Thus

where

We can use this technique to find confidence intervals for the regression coefficients b₀ and b₁ based on the t distribution with n–1 degrees of freedom. We can also calculate a confidence interval for the prediction of y for a given value of x in the same manner.

Examples

Example 1: Calculate the standard error for coefficients from Example 1 of Deming Regression Basic Concepts.

First, we define x̄_i, ȳ_i, u_i, v_i, r_i to be the same as x̄, ȳ, u, v, r from Property 1 of Deming Regression Basic Concepts except that the sample pair (x_i, y_i) is omitted.

We note that

Since

it follows that

and so the sum of squared data elements excluding x_i is equal to u + nx̄² – x_i², from which it follows that

which can be simplified to

Similarly

Since

it follows that

and so the sum product of data elements excluding x_i and y_i is equal to r + nx̄ȳ – x_iy_i, from which it follows that

which can be simplified to

Using the formulas

Using these formulas, we can calculate the standard errors of the intercept and slope for Example 1 of Deming Regression Basic Concepts as shown in Figure 1.

Figure 1 – Standard errors of coefficients

Here, the values for x̄, ȳ, u, v, r are shown in range E14:I14, and can be calculated exactly as shown in Figure 1 of Deming Regression Basic Concepts.

Cell E4 contains the formula =($A$13*E$14-B4)/($A$13-1). Note that it could also be calculated by the array formula

=AVERAGE(IF($A$4:$A$13=$A4,””,B$4:B$13))

Similarly, cell F4 contains the formula =($A$13*F$14-C4)/($A$13-1).

Cell G4 can be calculated by the formula

=G$14-$A$13*(E$14-B4)^2/($A$13-1)

or optionally by the array formula

=SUMSQ(IF(A$4:A$13=A4,””,B$4:B$13-E4))

Similarly, cell H4 can be calculated by the formula

=H$14-$A$13*(F$14-C4)^2/($A$13-1)

Cell I4 can be calculated by the formula =I$14-$A$13*(E$14-B4)*(F$14-C4)/($A$13-1) or optionally by the array formula =SUM(IF(B$4:B$13=B4,””,(B$4:B$13-E4)*(C$4:C$13-F4))).

Now we apply the jackknifing procedure. Cell J4 contains the formula =F4-K4*E4 and cell K4 contains the formula =($B$15*H4-G4+SQRT(($B$15*H4-G4)^2+4*$B$15*I4^2))/ (2*$B$15*I4). Finally, cells J16 and K16 contain =DEVSQ(J4:J13)*($A$13-1) and =DEVSQ(K4:K13)*($A$13-1). Cells J17 and L17 contain =SQRT(J16/$A$13) and =SQRT(K16/$A$13), yielding the standard error for the intercept of 1.3375 and for the slope of 0.2193.

The usual regression coefficient table can now be calculated as shown in Figure 2.

Figure 2 – Regression coefficient table

Note that we use df = n–2 = 10–2 = 8, although some implementations use df = n–1 = 9.

We see from Figure 2 that the slope coefficient is significant.

Observation: When the parameter θ is a smooth function, the delete-1 jackknifing approach described above does a pretty good job in computing the standard error, but when θ is not smooth (i.e. a small change in the sample can result in a large change in the estimate of θ), then this approach doesn’t work very well. An example of such a θ is the median. For such θ, delete-d jackknifing or bootstrapping work better.

Delete-d jackknifing approach

1. For each subset T with d elements from the sample, calculate θ_T-hat in the same way as θ-hat was calculated except that the elements in T are left out of the calculation.
2. Calculate the mean of the θ_T-hat, labeled θ-tilde, using the following formula where n = the sample size
3. The jackknifed estimate of the standard error of θ is now given by the formula

Note that some would replace n – d by n in the above formula.

In practice, it is best to choose a value of d between the square root of n and n.

Examples Workbook

Click here to download the Excel workbook with the examples described on this webpage.

Reference

NCSS (2016) Deming regression
https://www.ncss.com/wp-content/themes/ncss/pdf/Procedures/NCSS/Deming_Regression.pdf