Share this page

Identifying Errant Markers: Quality Assurance Systems in an E-Marking Environment

By Anne Pinot de Moira


The exploitation of innovative technology in the administration of high stakes national qualifications, has provided the opportunity to develop robust quality assurance systems to monitor and improve marking reliability.

This paper uses Monte Carlo simulation to explore two simple models of quality assurance: a hierarchical seeded system and a peerpair double marked system. The models suggest that item marking minimises the effect of systematic differences between markers but the extent to which this improves reliability at a paper level is dependent upon the number of items on a paper.

Beyond the positive effect of dividing a single paper among different markers, it is concluded that a quality assurance system which includes any element of sampling has little potential to influence marking reliability directly. It becomes a system for identifying errant markers for retraining. It is shown that, for a given set of quality assurance parameters, the peerpair double marking system is more likely to identify an errant marker than the hierarchical seeded system.

How to cite

Pinot de Moira, A. (2010). Identifying errant markers: quality assurance systems in an e-marking environment. Manchester: AQA Centre for Education Research and Policy. 


Connect with us

Contact our team

Join us

Work with us to advance education and enable students and teachers to reach their potential.

Apply now