Does the delivery matter? Examining randomization at the item level
Scales that are psychometrically sound, meaning those that meet established standards regarding reliability and validity when measuring one or more constructs of interest, are customarily evaluated based on a set modality (i.e., computer or paper) and administration (fixed-item order). Deviating from an established administration profile could result in non-equivalent response patterns, indicating the possible evaluation of a dissimilar construct. Randomizing item administration may alter or eliminate these effects. Therefore, we examined the differences in scale relationships for randomized and nonrandomized computer delivery for two scales measuring meaning/purpose in life. These scales have questions about suicidality, depression, and life goals that may cause item reactivity (i.e., a changed response to a second item based on the answer to the first item). Results indicated that item randomization does not alter scale psychometrics for meaning in life scales, which implies that results are comparable even if researchers implement different delivery modalities.
Item analysis, Randomization, Scales
Buchanan, Erin M., Riley E. Foreman, Becca N. Johnson, Jeffrey M. Pavlacic, Rachel L. Swadley, and Stefan E. Schulenberg. "Does the delivery matter? Examining randomization at the item level." Behaviormetrika 45, no. 2 (2018): 295-316.