Identifying Errant Markers: Quality Assurance Systems in an E-Marking Environment
PDF | 807.2 KB
Identifying Errant Markers: Quality Assurance Systems in an E-Marking Environment
Identifying Errant Markers: Quality Assurance Systems in an E-Marking Environment
Abstract
The exploitation of innovative technology in the administration of high stakes national qualifications, has provided the opportunity to develop robust quality assurance systems to monitor and improve marking reliability.
This paper uses Monte Carlo simulation to explore two simple models of quality assurance: a hierarchical seeded system and a peerpair double marked system. The models suggest that item marking minimises the effect of systematic differences between markers but the extent to which this improves reliability at a paper level is dependent upon the number of items on a paper.
Beyond the positive effect of dividing a single paper among different markers, it is concluded that a quality assurance system which includes any element of sampling has little potential to influence marking reliability directly. It becomes a system for identifying errant markers for retraining. It is shown that, for a given set of quality assurance parameters, the peerpair double marking system is more likely to identify an errant marker than the hierarchical seeded system.
How to cite
Pinot de Moira, A. (2010). Identifying errant markers: quality assurance systems in an e-marking environment. Manchester: AQA Centre for Education Research and Policy.