Estimation of composite score classification accuracy using compound probability distributions
Estimation of composite score classification accuracy using compound probability distributions
Estimation of composite score classification accuracy using compound probability distributions
Abstract
Presented is a demonstration of an intuitively simple, flexible and computationally inexpensive approach to estimating classification accuracy indices for composite score scales formed from the aggregation of performance on two or more assessments.
This approach uses a two stage application of the polytomous extension of the Lord-Wingersky recursive algorithm and can be driven by any IRT model with desired simplicity or required complexity to best represent the properties of the tests. The approach is demonstrated using operational data from a high stakes mathematics qualification which is formed from two tests administered on distinct occasions.
To provide the simplest representation of a test containing both dichotomous and polytomous items, the partial credit model is applied to model behaviour on the two tests. As an extension to this, a testlet model is applied to allow joint calibration of parameters from both tests.
This model provides more information to the calibration process at the expense of some added computational complexity. Further to this, the potential application of this approach in the absence of operational data is investigated using a comparison of simulated data to the observed data.
How to cite
Wheadon, C., & Stockford, I. (2013). Estimation of composite score classification accuracy using compound probability distributions. Psychological Test and Assessment Modeling, 55(2), 162–180.