Making an exam - a guide to creating a question paper video transcript
Making an exam - a guide to creating a question paper
A qualification is a way of demonstrating that someone has achieved a certain level of learning.
To gain a qualification, students often take examinations. The exams taken by GCSE and A-level students are based on a syllabus, which is now usually called a specification.
Specifications are created by exam boards and cover all the information and skills that students are expected to have learnt by the end of their course. They are accredited by the exams regulator, Ofqual, to make sure that they meet the national criteria for the qualification.
A question paper tests how well a student can demonstrate that they have acquired the knowledge, understanding and skills set out in the specification.
As it would be impossible to test the entire specification in one set of exams,
we select which areas to focus on in a particular paper, and ensure that all parts of the specification are covered over a five year period.
At the same time - as we develop the question paper - we also create the marking guidance , or mark scheme , that goes with it. This ensures that the overall assessment is valid and students will be given the right marks for their answers
Creating a question paper takes a lot of time. In fact, work starts more than a year and a half before students are due to sit the exam.
A question paper is put together by an exam committee. The members of the committee are often teachers, who have developed assessment expertise as a result of working as examiners. The committee is made up of senior examiners including the chair of examiners for the subject, a chief examiner for the specification and a lead assessment writer for each question paper.
There is also a qualification developer, who is responsible for managing the process and ensuring that the question paper complies with AQA’s rules. In addition, we draw on the research and assessment expertise of our Centre for Education Research and Analysis.
There are two other types of specialists that help ensure the quality and consistency of the question paper – the reviser and the scrutineer. We’ll come back to them in a moment.
Before we start to write a question paper, the chair and chief examiner review the papers from the previous exam series. They look in detail to see how well each question performed. For example – did students understand what they were being asked to do? This review is one of the ways we ensure that we continuously improve the quality of our question papers. The lead assessment writer then develops the blueprint for the paper with the chief examiner. This is one of the most important parts of the whole process, as all the individual questions will be checked carefully against the blueprint.
Once a question and its mark scheme have been created, it is reviewed by the reviser and other senior examiners. They use a comprehensive checklist to make sure each question offers the right level of challenge, tests content that is covered in the specification and is error free.
We want to ensure that our questions work for everyone, so we have a team that modifies papers for visually impaired students. We also work with The British Association of Teachers of the Deaf, who tell us if certain words or images could make the question more difficult for some students.
When all of these checks have been completed, the individual questions are pulled together into a question paper.
At this stage there are a number of further checks to be made. The paper is read by the qualifications developer and professional proofreaders. They check the spelling, grammar and accuracy of the questions, as well as any images, diagrams and charts. After that, the lead assessment writer and reviser check that the paper follows the blueprint.
The next stage is where the first scrutineer comes in. Our scrutineers are subject experts who sit the paper exactly as a student would. They check to see that all the questions are clear, the paper offers the right level of challenge and that it can be completed in the allocated time.
The question paper is then reviewed by the Question Paper Approval Committee, who sign it off as fit for purpose and error free. It’s proofread again and a second scrutineer sits the paper. They check it against the mark scheme, as we need to know if they’ve given the answers we would expect.
A member of the subject team who hasn’t seen the paper before gives it a final check. If everything is fine, the chair of examiners gives their approval.
After 18 months of work, the question paper is ready to be sent to the printers. When we get the printed papers back, we do one last check.
But that’s not quite the end of the process. The question paper now goes back to the specialist team who produce all the different versions of the paper - including large print and braille – which will make it accessible to visually impaired students.
Making an exam - a guide to creating a question paper, brought to you by AQA. For more information visit aqa.org.uk.