Examinee Variance in Strategy Shifts During Testing: An Explanatory IRT Approach

Abstract

Considerable work (both within the psychometric and cognitive literature) has explored the tendency for response strategies to shift within the course of a single assessment. The current project proposes a novel, flexible item response model that intends to capture the pattern of these strategy shifts, as well as any latent classes defined by differences in these shifting patterns. The novel model represents an integration of hidden Markov techniques within the framework of Explanatory Item Response Modeling. The feasibility of such a modeling approach is evaluated via parameter recovery with a set of Monte Carlo simulations, and its practical utility demonstrated with an empirical example involving response data from a spatial reasoning task. While structural parameters are generally estimated well, much caution should be exercised when interpreting person level parameters.