The Achieved Weightings of Assessment Objectives as a Source of Validity Evidence
In its proposed framework for validating Ofqual-regulated assessments, Ofqual identifies four key areas of evidence on which validity arguments should be based. One key area is the alignment between assessment and the curriculum/syllabus.
The work reported here demonstrates a method for producing a source of validity evidence that falls under this category: the comparison of the intended and achieved weightings of assessment objectives at qualification and question paper level. Screening data was produced at unit-subject level for the majority of AQA GCSE and A-level specifications while six individual specifications were further analysed at the question-unit level.
Where problems with achieved weightings occurred, some possible issues – both general to common assessment structures and paper-specific – were identified that could be fed back into the specification design and examination paper-writing processes. The complexities of interpreting and improving the achieved weightings of assessment objectives, in a context in which they are only one of a number of interrelated facets of validity, are discussed.