Variation in respondent speed and its implications: Evidence from an adaptive testing scenario

Abstract: The more frequent collection of response time data is leading to an increased need for an understanding of how such data can be included in measurement models. Models for response time have been advanced, but relatively limited large-scale empirical investigations have been conducted. We take advantage of a large dataset from the adaptive NWEA MAP Growth Reading Assessment to shed light on emergent features of response time behavior. We identify two behaviors in particular. The first, response acceleration, is a reduction in response time for responses that occur later in the assessment. We note that suchreductions are heterogeneous as a function of estimated ability (lower ability estimates are associated with larger increases in acceleration) and that reductions in response time lead to lower accuracy relativeto expectation for lower ability students. The second is within-person variation in the association be-tween time usage and accuracy. Idiosyncratic within-person changes in response time have inconsistent implications for accuracy; in some cases additional response time predicts higher accuracy but in many cases additional response time predicts declines in accuracy. These findings have implications for models that incorporate response time and accuracy. Our approach may be useful in other studies of adaptive testing data.