Rasch Analysis Basic Concepts

Introduction

In addition to classical test theory (CTT), as described elsewhere on this website, there is a newer approach, called item response theory (IRT). IRT is a psychometric approach that focuses both on the subjects’ responses to a test item as well as the qualities of the test items.

In general, a subject’s ability to answer a question (item) correctly depends on the subject’s ability and the difficulty of the question. The easier the question and the more ability of the subject, the higher the likelihood that the subject will answer the question correctly.

Notation

We use subscripts i for the items and s for subjects. We express the scores that the subjects received on each item as either a 1 (for correct) or 0 (for incorrect). Thus, the score that the subject s got on item i is expressed as x_si, which takes the value 0 or 1. The matrix X = [x_si] expresses the scores of all the subjects on all the items.

What we are really interested in, however, is to determine the overall ability level β_s of each subject and the difficulty of each item δ_i. Here, we assume that a higher value of δ_i means that item i is more difficult, and a higher value of β_s means that subject s is more able.

Rasch approach

In an ideal world, we would arrange the items in order from least difficult to most difficult in a way in which a subject s would answer items 1 through i correctly (for some i) and items from i + 1 on incorrectly. In this case, the ability of subject s is δ_i. Similarly, if we arranged the subjects in order of ability, we could determine the difficulty of an item as β_s where subjects 1 through s could answer that item correctly, while none of the subjects after s could answer the item correctly.

Since this ideal situation doesn’t occur in the real world, some adjustments are necessary, but essentially this is the motivation behind the Rasch approach, which is based on the following formula:

which says that the probability that student s answers item i correctly is equal to the formula on the right which is a function of β_s – δ_i, that is the difference between student s’s ability and item i’s difficulty.

Logit function

We define the logit function as follows:

This function has the desirable property that it transforms y in the range -∞ to +∞ to logit y in the range 0 to 1, as shown in Figure 1.

Figure 1 – Logit function

Thus, the Rasch model is based on the formula

Observations

Notice that when β_s = δ_i, the probability that subject s answers item i correctly is .5. When β_s > δ_i, then this probability is greater than .5, and when β_s < δ_i, this probability is less than .5. E.g. if β_s = 1 and δ_i = -1, then the probability that subject s answers item i correctly is

Notice too that for any i

Similarly

and so

which proves that

Since this equality holds for any item i, it also follows that β_s – β_r = the log of the ratio of the number of correct answers by subject s plus the number of incorrect answers by subject r divided by the number of incorrect answers by s plus the number of correct answers by r. In fact, this statement should hold for any subset of the items, not just one item or all the items.

Thus, we can express the difference between the ability of any two subjects in a way that doesn’t involve the difficulty of the items. Similarly, we can express the difference between the difficulty of any two items in a way that doesn’t involve the abilities of the subjects.

Examples Workbook

Click here to download the Excel workbook with the examples described on this webpage.

References

Moultan, M. H. (2003) Rasch estimation demonstration spreadsheet
https://www.rasch.org/moulton.htm

Wright, B. D. and Stone, M. H. (1979) Best test design. MESA Press: Chicago, IL
https://research.acer.edu.au/measurement/1/

Wright, B. D. and Masters, J. N. (1982) Rating scale analysis. MESA Press: Chicago, IL
https://research.acer.edu.au/measurement/2/

Boone, W. J. (2016) Rasch analysis for instrument development: why, when, and how?
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5132390/pdf/rm4.pdf

Boone, W. J. and Noltemeyer, A. (2017) Rasch analysis: A primer for school psychology researchers and practitioners. Cogent Education
https://edisciplinas.usp.br/mod/resource/view.php?id=3333001

Furr, M. and Bacharach, V. R. (2007) Psychometrics: an introduction; Chapter 13: Item response theory and Rasch models. Sage Publishing
https://in.sagepub.com/sites/default/files/upm-binaries/18480_Chapter_13.pdf

Wright, B. and Stone, M. (1999) Measurement essentials, 2^nd ed.
https://www.rasch.org/measess/