Gains in marking reliability from item-level marking: is the sum of the parts better than the whole?
Marking of high-stakes examinations in England has traditionally been administered by schools and colleges sending their examination papers directly to examiners. As a consequence, the work of one candidate has, historically, been marked by one examiner, as has work of an entire centre.
Previous studies have suggested that the marking of both whole scripts and whole centres is liable to bias, caused either by examiner characteristics or the characteristics of the allocation of marking.
This study used operational data from two Geography papers which had moved from whole script, whole centre marking to item-level marking, and then back again, to quantify the gains in reliability from item-level marking. It found that there were substantial gains in the reliability of marking for the highest performing candidates when the marking was at item level. The reasons for these gains did not appear to be associated with any characteristics of a centre entry.