Fully Processed candidates: an analytical solution to the Post-Awarding Drift problem

Abstract

The number of candidates for whom mark data is available at the time of an award is less than the total entry in the vast majority of cases — the relative difference between these is referred to as the Percentage Fully Processed (PFP).

At some point between award and publication of results, the mark data for the additional candidates will become available and will have some effect on the percentage outcomes at each grade — the outcomes will drift, known as Post-Awarding Drift (PAD). This effect has been the focus of previous studies, where empirical and simulation analyses were undertaken.

These studies found some relationship between the PFP and PAD, which further resulted in a recommendation of lowering the minimally required PFP (MRPFP) at award from 85% to 70% across all specifications in order to avoid PAD (although 75% was ultimately chosen).

The aim of this study is to investigate PAD from an analytical perspective in order that this MRPFP might be chosen in a more substantiated way. The results show that the PAD as observed empirically or by simulation can be analytically explained, at the expense of making some practical assumptions, and that the MRPFP will vary from unit to unit. This is mainly driven by the size of the total entry so that in the extreme cases of very large-entry units (e.g. total entry of 400,000) the MRPFP is less than 1%.

Additionally, the methodology was explored during the summer 2012 awarding series with the primary aim of facilitating decisions being made about units with low PFPs. Discussion is also provided around the use of additional information that is available during award preparation that can better inform the decisions made around the MRPFP.

How to cite

Hales, R. (2012). Fully Processed candidates: an analytical solution to the Post-Awarding Drift problem. Manchester: AQA Centre for Education Research and Policy.

Keywords

• Transparency