14. Violations of the Assumption of Independence I—Multidimensionality and Response Dependence

Local Independence

The statistical independence formalized in the Rasch model reflects the intentions of test developers when constructing and assembling items. The same independence is implied in CTT . The Rasch model is typically used for analyses of the psychometric properties of scales or tests in which responses to a number of different items are summed. They are summed because they are considered to capture a unidimensional construct. The summed responses of more than one item should be more valid and reliable than a response to one item only. However, this is true only when each item measures the same trait as the other items in the scale and provides some unique information not provided by the other items. In other words, in order to provide a reliable scale of summed items each item needs to provide related but independent information, or relevant but not redundant information. An analysis according to the Rasch model will reveal, as an anomaly , items that do not provide relevant or independent information.

The dichotomous Rasch model is

(14.1)

where is the integer response variable for person n with proficiency responding to item i with difficulty . The model implies a single dimension with values of and located additively on the same scale .

The model also implies statistical independence of responses in the sense that

(14.2)

where denotes the matrix of responses . That is, the probability of answering the set of items correctly equals the product of the probabilities of answering the individual items correctly.

The holding of Eqs. (14.1) and (14.2) together is generally referred to as local independence (Lazarsfeld & Henry, 1966; Andrich, 1991). The term local refers to the idea that all the variation among responses to an item is accounted for by the person parameter , and therefore that for the same value of , there is no further relationship among responses.

In the Rasch model, the person parameter is the source of general dependence among responses to items in the sense that a person with a high value of will tend to respond positively to all items, and the opposite for a person with a low value of . In the estimation of the item parameters, can be eliminated. With this parameter eliminated, or for the same value of , there should be no further relationship among the items. The absence of this kind of relationship is referred to as local independence .

Two Violations of Local Independence

Local independence in Rasch models defined as above can be violated in two generic ways. First, there may be person parameters other than that are involved in the response. This is a violation of unidimensionality and therefore statistical independence relative to the model of Eq. (14.1).

Second, for the same person and therefore the same value of , the response to one item might depend on the response to a previous item. This is a violation of statistical independence relative to Eq. (14.2). To distinguish this latter violation of Eq. (14.2) from the violation of unidimensionality, we refer to the latter as response dependence . Both these violations have been formalized algebraically in Marais and Andrich (2008a, b). The papers also provide some examples of these two types of violations of independence in practice.

Multidimensionality

Many scales in psychology, education and social measurement in general, which are constructed to measure a single variable, are nevertheless composed of subsets of items which measure different but related aspects of the variable. An example is the Functional Independence Measure (FIM™) motor scale (Keith, Granger, Hamilton, & Sherwin, 1987), which consists of 13 items, ranging from bladder management to climbing stairs. These items can be grouped into subsets, for example, Sphincter Control can comprise Bowel Management and Bladder Management. Although the presence of subsets captures better the complexity of a variable and increases its validity , it compromises the model’s unidimensionality. Another example is the Australian Scholastic Aptitude Test (ASAT) where items, which are summed, are grouped into subsets representing mathematics, science, humanities and social science (Bell, Pattison, & Withers, 1988).

Multidimensionality is also found in items that are linked by attributes such as common stimulus materials, common item stems, common item structures or common item content. These have been described as subtests (Andrich, 1985), testlets (Wang, Bradlow, & Wainer, 2002) or item bundles (Rosenbaum, 1988; Wilson & Adams, 1995).

Formalization of Multidimensionality

Marais and Andrich (2008b) formalized multidimensionality in the following way. Consider a scale composed of s = 1, 2, …, S subsets and

(14.3)

where , is the common trait for person n among subsets and is the same variable as in Eq. (14.1), is the distinct trait characterized by subset s and is uncorrelated with .

Therefore, is the value of the main, common variable or trait among subsets, and is the variable or trait unique to each subset. The value characterizes the magnitude of the variable of subset s relative to the common variable among subsets.

Consider Figs. 14.1 and 14.2. In Fig. 14.1 all six items measure the variable . In Fig. 14.2 all six items measure the common variable , but in addition, items 1–3 also measure the unique variable and items 4–6 also measure the unique variable .

The design for S subsets, each with I items, is summarized in Table 14.1.

Table 14.1

Summary of subset design

Items	Subsets
	1	2	…	S
1			…
2			…
⋮	⋮	⋮	⋮	⋮
I			…

Response Dependence

A second violation of the assumption of independence is response dependence . Response dependence occurs when a person’s response to an item depends on the person’s response to a previous item. This can occur in cases where a correct answer on a question gives a clue or the answer to one or more subsequent questions. Or the case where the structure of different questions is such that an answer to one question logically implies the answer to another question. Kreiner and Christensen (2007) show that the items Climbing one flight of stairs and Climbing several flights of stairs of the physical functioning subscale of the SF-36, a widely used rating scale in health research, are response dependent. Similarly, the items Walking one block, Walking several blocks and Walking more than a mile are dependent in this way. They should be different levels of one ordered category item. Another example is where judges make decisions on multiple criteria with respect to some object and a halo effect operates across all criteria.

Formalization of Response Dependence

The statistical independence of the model of Eq. (14.1) implies that

(14.4)

Marais and Andrich (2008a) formalised response dependence of item j on item i by

(14.5)

Equation (14.5) does not satisfy Eq. (14.4). The value d characterizes the magnitude of response dependence . A correct response = 1 by person n to item i reduces the difficulty of item j to thus increasing the probability of the same correct response = 1 to item j. Similarly, the response = 0 on item i increases the difficulty of item j to , thus increasing the probability of the same incorrect response = 0 to item j.

Estimating the Magnitude of Response Dependence

Andrich and Kreiner (2010) describe a way of estimating the value of d in Eq. (14.5), which characterizes the magnitude of response dependence . It is estimated as a change in the location of the difficulty of item j caused by its dependence on item i. The focus is on dependence between two dichotomous items.

According to the procedure described by Andrich and Kreiner (2010), item 5 was resolved into two distinct items, now called items 5S0 and 5S1. Item 5S0 is composed of responses to item 5 from those persons whose responses on item 4 are 0. Item 5S1 is composed of responses to item 5 from those persons whose responses on item 4 are 1. Because these two items are still dependent on item 4, item 4 is deleted from the analysis. Table 14.5 shows the individual item estimates and fit statistics for these two items. It is clear from Table 14.5 that these two items have very different difficulty estimates. The above steps can be carried out routinely in RUMM2030. The difference in their estimates gives an estimate of the magnitude of response dependence between items 4 and 5, according to Eq. (14.6).

Table 14.5

Individual item fit statistics for items 5S0 and 5S1 in the case of dependence

Seq	Item	Type	Location	SE	FitResid	DF	ChiSq	DF	Prob
31	005S0	Poly	−0.618	0.196	−0.973	171.68	11.779	9	0.226
32	005S1	Poly	−4.865	0.296	−0.294	763.87	6.202	9	0.720

(14.6)

The estimated magnitude of = (−0.618 – (−4.865))/2 = 2.12 is very close to the value of used to simulate the data set.

Table 14.6 shows the individual item estimates and fit statistics for items 5S0 and 5S1 when exactly the same data set was simulated with no dependence between items 4 and 5. It is clear that the resolved items have very similar estimates. Once again, the estimated magnitude of = (−2.507 – (−2.355))/2 = 0.08 is very close to the value of used to simulate the data set without response dependence .

Table 14.6

Individual item fit statistics for items 5S0 and 5S1 in the case of no dependence

Seq	Item	Type	Location	SE	FitResid	DF	ChiSq	DF	Prob
31	005S0	Poly	−2.507	0.181	−1.270	172.64	7.171	9	0.619
32	005S1	Poly	−2.355	0.123	−1.088	763.87	8.270	9	0.507

The Effects of Violations of Independence

Although some have concluded that, in the specific situation they describe, violations of independence did not have big effects on estimates (e.g. Smith, 2005 cited in Marais & Andrich, 2008a), we have found significant effects. These effects are described in Marais and Andrich (2008a, b). Figure 14.3 shows person and item distributions from these simulation studies. Data were simulated with no dependence. Data were also simulated according to the same specifications but with either response dependence or multidimensionality . It is clear from these person distributions that, relative to the condition with no dependence, the variance increased with response dependence and decreased with multidimensionality . In summary, in these simulation studies response dependence resulted in increased reliability and increased variance of person estimates. Multidimensionality resulted in decreased reliability and decreased variance of person estimates. These inferences could be made because the properties of the data were known from the simulations. In real data, professional judgement from multiple pieces of evidence in context is required to decide the source of dependence.