The effects of consistency of performance on A-level examiners' judgements of standards
One source of evidence used for the setting of minimum marks required to obtain grades in General Certificate of Education (GCE) examinations is the qualitative, expert judgement of examiners.
The effect of consistency of candidates' performance across questions within an examination paper upon examiners' judgements of gradeworthiness was investigated, for A-level examinations in two subjects. After controlling for mark and individual examiner differences, significant effects of consistency were found. The pattern of results differed in the two subjects. In Biology, inconsistent performance produced lower judgements of grade-worthiness than consistent or average performance. In Sociology, very consistent performance was preferred over average consistency.
The results of this study showed that a feature of the examination performance that was not part of the marking scheme affected grading decisions.
It is concluded that examiners' judgements of standards should be supported by other sources of evidence, such as examination statistics.