Dunn’s Test after KW

Basic Concepts

Dunn’s test is similar to Nemenyi’s test, except that it relies on the normal distribution and includes a ties correction.

Dunn’s test uses the statistic

where

and the standard error is

Here n = the total sample size and n_i and n_j are the sizes of the groups being compared.

If there are a lot of ties, an improved version of the standard error is given by

where f is as in the ties correction for the Kruskal-Wallis test. This test is equivalent to

where z_crit = NORM.S.INV(α*/2).

Familywise error correction

Here, α* is the significance level using some form of familywise error correction. E.g. if you want to account for all possible pairwise comparisons using a Bonferroni-like correction, then you would set α* = α/(k(k-1)/2) where k = the number of groups. Note that if you know in advance that only certain pairwise tests will be interesting, then you can reduce the correction factor. E.g. if α = .05 and k = 4, then α* = .05/6 = .0833. If you are certain (before looking at the data) that you will only perform comparisons of groups A vs. B and A vs. C, then you can use α* = .05/2 = .025 instead.

When a Bonferroni correction is used this test is quite conservative.

Data Analysis Tool

Real Statistics Data Analysis Tool: The Single Factor Anova data analysis tool can be used to perform Dunn’s test, as shown in Example 1.

Example 1: Conduct Dunn’s Test for Example 1 of Kruskal-Wallis Test to determine which groups are significantly different.

The procedure is the same as for Example 1 of Kruskal-Wallis Test, except that this time the Dunn option is selected in the dialog box shown in Figure 1 of Single Factor Anova Analysis Tool. After clicking on the OK button the output shown in Figure 1 appears.

Figure 1 – Dunn’s Test

Since .011785 < .016667, we see there is a significant difference between the New and Old creams; similarly, since .01082 < .016667, we conclude there is a significant difference between New and Control. There is no significant difference between Old and Control.

Some key formulas from Figure 1 are shown in Figure 2.

Cells	Item	Formula
S3	α*	=R3/(COUNT(P5:P7)*(COUNT(P5:P7)-1)/2)
O5	R1	=RANK_SUM(B4:D13,1,1)
P5	n1	=COUNT(B4:B13)
S5	R1 mean	=O5/P5
I8	n	=SUM(P5:P7)
R8	z-crit	=NORM.S.INV(1-R3/2)
P11	diff means	=ABS(Q5-Q6)
Q11	std err	=SQRT((P$8*(P$8+1)-TiesCorrection($B$4:$D$13)/(P$8-1))/12*(1/P5+1/P6))
R11	z-stat	=P11/Q11
S11	R-crit	=Q11*R$8
T11	p-value	=2*(1-NORM.S.DIST(R11,TRUE))

Figure 2 – Selected formulas from Figure 1

Note that the values of z-crit and R-crit in Figure 2 are based on the alpha value of .05, and not the corrected .016667 value.

Examples Workbook

Click here to download the Excel workbook with the examples described on this webpage.

References

NCSS (2012) One-way analysis of variance
https://www.ncss.com/wp-content/themes/ncss/pdf/Procedures/NCSS/One-Way_Analysis_of_Variance.pdf

Stack Overflow (2020) Which post-hoc test should I conduct after doing a Kruskal-Wallis?
https://stackoverflow.com/questions/31434166/which-post-hoc-test-should-i-conduct-after-doing-a-kruskal-wallis

Dinno, A. (2015) Nonparametric pairwise multiple comparisons in independent groups using Dunn’s test
https://journals.sagepub.com/doi/pdf/10.1177/1536867X1501500117