Who is the specialist? The effect of specialisms on the marking reliability of an English literature examination
The effect of examiner specialisms on marking reliability is an under-researched topic. This report explores the topic using multilevel modelling of mark remark data from quality monitoring samples. It aims to provide an answer to the question: do examiners mark more accuratelywhen they mark responses from their own specialisms?
Data from 53 examiners across more than 3,000 responses to optional questions from an A-level English literature examination were modelled. The optionality in the question paper was based on prescribed texts. Each examiner was matched to a home centre. The texts offered by a home centre (based on candidates’ choices in the examination) became the examiner’s specialisms.
The report draws one main conclusion. Examiners who have experience within their working environment of a range of specialisms, mark more reliably than those examiners who work with a limited number of specialisms. This finding was statistically significant. Suggestions for future work in this area are provided at the end of the report.