Who is the specialist? The effect of specialisms on the marking reliability of an English literature examination

Abstract

The effect of examiner specialisms on marking reliability is an under-researched topic. This report explores the topic using multilevel modelling of mark remark data from quality monitoring samples. It aims to provide an answer to the question: do examiners mark more accuratelywhen they mark responses from their own specialisms?

Data from 53 examiners across more than 3,000 responses to optional questions from an A-level English literature examination were modelled. The optionality in the question paper was based on prescribed texts. Each examiner was matched to a home centre. The texts offered by a home centre (based on candidates’ choices in the examination) became the examiner’s specialisms.

The report draws one main conclusion. Examiners who have experience within their working environment of a range of specialisms, mark more reliably than those examiners who work with a limited number of specialisms. This finding was statistically significant. Suggestions for future work in this area are provided at the end of the report.

How to cite

Whitehouse, C. (2014). Who is the specialist? The effect of specialisms on the marking reliability of an English literature examination. Manchester: AQA Centre for Education Research and Practice.

Keywords

• Reliability
• Examiners