Can we predict who will be a reliable marker?

Abstract

AQAs, Research and Policy Analysis Department have conducted a large number of studies that have attempted to identify factors which might allow awarding bodies to predict those examiners who are likely to mark most reliably and those who are likely to require additional training or monitoring. Most of the work, however, is not publicly available.

This paper provides a review of this research and internationally published studies relevant to predicting marker reliability. The relative importance of the following variables are discussed: examining experience; teaching experience; subject knowledge; senior examiners’ ratings of examiners’ previous performance; and examiner traits such as logical reasoning capacity and personality.

Overall the conclusion of the review is that the criteria used by UK awarding bodies to select examiners (subject knowledge and teaching experience) are not empirically supported. The paper ends with the description of work that is currently being undertaken to examine whether psychometric measures of personality can be used to predict the marking reliability of individuals with distinctly different levels of examining experience, teaching experience and subject knowledge.

How to cite

Meadows, M. (2006). Can we predict who will be a reliable marker? Manchester: AQA Centre for Education Research and Policy.

Keywords

• Reliability
• Marking