Awarding in the 21st century - a virtual model
PDF | 617.1 KB
Awarding in the 21st century - a virtual model
Awarding in the 21st century - a virtual model
By Lesley Meyer, Jo-Anne Baird, Neil Stringer, Lynne O'Sullivan, Carolyn Adams
Abstract
Although hundreds of thousands of markers are recruited internationally to mark examinations, little research has been conducted on the selection criteria that should be used. Many countries insist upon teaching experience for markers and this has frequently become embedded in the cultural expectations surrounding the tests.
Shortages in markers for some of the UKs national examinations has led to non-teachers being hired to mark certain items and changes in technology have fostered this approach. For a national curriculum English examination taken at age 14, this study investigated whether teaching experience is a necessary selection criterion.
Fifty seven markers with different backgrounds were trained in the normal manner and marked the same 98 students work. By comparing the marking quality of graduates, teacher trainees, teachers and experienced markers, this study shows that teaching experience was not necessary for most of the examination questions. Small differences in inter-rater reliability estimates on the Shakespeare reading and writing tasks were found, such that non-teachers were less reliable than those with teaching experience. A model for selection of markers to mark different question types is proposed.
How to cite
Meyer, L. et al (2006). Awarding in the 21st century - a virtual model. Manchester: AQA Centre for Education Research and Policy.