Professional Development

Hi. My name's Tom Atkinson, Chief Examiner for A-level PE, and also, an item writer for GCSE PE. Today's session is going to look at assessment principles, and hopefully build on your existing knowledge of our courses. We hope to give you an insight into how our assessments and mark schemes are designed.

The principles of assessment of the factors we must consider when producing a valid assessment. A valid assessment is one that measures what it intends to measure, and only that. In our case, this is a student's ability in GCSE or A-level PE. There are several factors and principles that contribute to assessment validity, and these are displayed on the screen. We will touch on all of these factors throughout the session, making links with the different question types that exist across physical education assessments.

The assessment objectives which were introduced with the new specifications in 2016, underpin the validity of our assessment. As you will know, physical education exams assess students across three areas of assessment. AO1 assesses students' knowledge and understanding of the factors that underpin performance and involvement in physical activity, in sport. These are facts and things which do not change. AO2 tests candidates' ability to apply their knowledge and understanding to a novel or unique sporting situation. And finally, AO3 requires students to showcase the skills of analysis and evaluation. It's worth noting at this point that we will not be covering AO4 in this training, as this AO relates only to the non-exam assessment, otherwise known as the NEA.

The reason why these assessment objectives are so important is that they allow us to produce comparable assessments, year on year. By ensuring that we adhere to the same percentages each year, or across exam papers, we can ensure that, broadly speaking, every assessment is of equal demand. The three assessment objectives also provide examiners with the opportunity to differentiate between students of varying abilities.

Take a minute to look at the three questions currently on the screen, making a note of which assessment objective each is examining. Pause the video now if you would like to complete this task in your own time. As can be seen, Question One was a 'Define' question. The definition of something as a fact that doesn't change is therefore assessing the A01. Question Two is also assessing AO1. It is a common misconception that questions focus on the changes which occur during exercise, at AO2. This is not the case, as these changes are consistent across all forms of exercise or activity. For a question to be deemed AO2, knowledge and understanding must be applied to a novel or unique sporting situation. Finally, Question Three would be assessing the candidates’ ability to analyse and is therefore AO3.

The first question type we are going to look at is multiple-choice questions. Multiple-choice questions make up the first 5 to 7 questions on a GCSE paper, and the first two questions in each section on the A-level papers. Multiple-choice questions in this specification can target either AO1 or AO2. They consist of a question or incomplete statement to be completed and a number of options from which the students will select the correct answer. The correct answer is known as the 'Key', and the incorrect answers are called 'distractors'. Three or four options are usually provided, including the Key. These options are presented in alphabetical or numerical order.

There are several factors which must be considered to write an effective multiple-choice question. These include ensuring that there is only one correct answer with no ambiguity about which it may be. Using negative terms, i.e., 'not' with care, and avoiding double negatives. Ensuring that correct responses are not consistently shorter, or longer, or more detailed than the distractors. And finally, making sure all the distractors are plausible. Research suggests that the plausibility of the distractors is more important than the number of options available. A plausible distractor is one that cannot be eliminated by a student that lacks the relevant subject-specific knowledge. If implausible, distractors could be eliminated, solely on common sense of everyday knowledge. This is why when no more than two plausible distractors can be offered, options are limited to three. In this case, another fourth option would simply add to the student's reading time but would not be used given its implausibility.

By following these principles, teachers could write their own multiple-choice questions, which allow them to quickly assess a large number of students. If carefully considered, they can be used as hinge questions to identify students who are making good progress, and those who require additional support. Finally, multiple-choice questions offer an opportunity to quickly identify and eliminate common misconceptions. This will be achieved by basing one of the distractors on the misconception being targeted.

The next type of question we'll be looking at is short answer questions. These are the most flexible questions on the physical education assessments, in that they can cover one or more of the assessment objectives. And their mark tariffs vary from 1 to 6 marks. These types of questions are predominantly marked using a points-based mark schemes. Although level of response mark schemes can be used, if this would be a valid way of assessing the skills required by the question and facilitate more accurate marking. I will address the different types of mark schemes we use later in this presentation.

Short answer questions make up the vast majority of marks on a GCSE PE paper and account for 10 marks per section in A-level PE. The choice of command word plays a key role in telling students what the task is, i.e., the type, and sometimes the depth of response that is required. Due to the fact that the short answer questions in PE can cover any of the three assessment objectives, the use of the correct command words is an important way of sign-posting the students, what type of response we are looking for. A list of the command words for GCSE and A-level is available on the AQA website, but there are some which are used more frequently than others.

The table on screen shows some of the most commonly used command words across GCSE and A-level PE. Students should not only be familiar with these command words but understand what is expected of them when they are seen. 'State' and 'Name' both assess students' knowledge and only require answers of a few words. 'Outline' and 'Describe' also assess students' knowledge, but require a more detailed and structured response. The command word 'Explain' requires students to set out a purpose or reason, eg why or how. Connectives like 'because' can, therefore, be useful to students when forming their response. 'Explain' in PE is often, though not always used for AO2 questions, which require students to apply their knowledge and understanding to a novel or unique sport situation. In these application questions, students must link their response back to the specific context in order to access the marks.

In the example question shown, simply stating a functional characteristic of Type Two muscle fibers, such as high contraction force would not be creditworthy. If the candidate correctly explained, however, that a high contraction force helps a 100m sprinter because it allows them to make a fast start in the race, this would be awarded a mark. Finally, the AO3 command words are 'Analyse' and 'Evaluate'. 'Analyse' questions require students to break down large topics and give a detailed analysis of how each part impacts on the performance or situation provided in the question. 'Evaluate' requires students to present advantages and disadvantages relevant to the specific scenario, and their impact on performance.

To access full marks, students will be required to consider both sides of an argument. Extended response questions play a role in our assessments at both GCSE and A-level. GCSE PE papers end with two extended response questions, one 6-mark, and one 9-mark question. However, extended response questions make up approximately two-thirds of the available marks on A-level PE exams, with each section containing one 8-mark, and one 15-mark question. These questions will always cover AO1, AO2, and AO3, with fixed weightings, and are designed to be more open and allow students to answer them in a number of different ways. This is why these questions are marked holistically using level of response mark schemes. Across both specifications, these questions can include a synoptic element.

Nine-mark questions in GCSE aim to assess the specification content from across the paper unless they're Question Six. Students are given the opportunity to showcase their knowledge, skills, and understanding from across the specification in their answer. At A-level, two 15-markers and one 8-marker across each exam series, i.e., both papers are synoptic. Combining knowledge of content from two different units on the course. They can cross papers, so Paper One synoptic question can examine Paper Two content, for example.

The choice of language in questions requires some important considerations. We're going to explore some of these considerations now. When creating assessments, it is important that we carefully consider the language you use to ensure students understand what the question is asking. Questions should be concise and aimed at a suitable level of reading demand for the qualification level, i.e., GCSE or A-level. The choice of language used needs to make the assessments accessible to all students. On the screen, you can see two example exam questions. In each example, the language could be improved to make the questions more accessible to students. Take a minute now to consider any improvements you might make. You might wish to pause the video at this point if you wish to complete this task in your own time.

In the first example, the terminology has been changed to match that of the specification. It is important to ensure that teaching and assessment resources reflect the language in the specification and subject-specific vocabulary. This is what students will see in exams and will be published in mark schemes for examiners to use. In the second example, the question has been rewritten using more accessible language and structure. The addition of a 'stem', which is an introduction to the question, and bullet points, should help students unpick the question to better ascertain what they are required to do.

Physical education exams must focus on specific sports. This can cause issues with biases but is unavoidable. Where possible, examples can be given from a spot of your choice. However, this is rarely practical from a writing and marking reliability perspective. It also adds to the cognitive loads for students. Football, athletics, and tennis are all named in specific sections of the A-level specification, requiring this content to be specifically linked to these sports. However, more generally, students should be able to apply their knowledge to a range of activities, including endurance sports, team games, aesthetic sports, and target sports. Some commonly used examples, of which are included on screen now.

While there is an expectation that students have a knowledge of a range of sports, we do understand that sometimes a prompt could be helpful. With this in mind, context and imagery can be used to provide students with a hook into the question. It is important to ensure that when used, images are accurate and ambiguous. Irrelevant information should not be included as this may result in the wrong information being used. If an image is not strictly needed to the question, the writer will need a balance advantage that it may make the question seem less daunting, against the possible risks that part of the text may not be read thoroughly, or that a student may be led astray by an element in the image that was not intended to be important.

In the example shown on the left, an image of a track cyclist is provided to differentiate from road cycling. As a high-demand question on a specification topic, we wanted to ensure that we examined students on their knowledge of biomechanics, and not just for a question. Similarly, on the right, while many students will know what golf is, applying it to Schmidt’s Schema Theory requires them to have some understanding of the different environments which golfers may find themselves in. As such, images were provided to give them some examples.

The following slides contain three examples of questions which may or may not require editing. Take some time to look at this item and consider what feedback you would offer to the writer to improve it. There are several issues which could be picked up here. The first is the lack of a command word. While we do see some multiple-choice questions start with words such as 'which', in short answer and extended response questions, a command word is important to help students understand what is required of them. The command word should be the first word in the question, after a stem, if one is needed. In this case, adding 'Evaluate' at the start of the question clearly indicates to the students that this is an AO3 question, as well as needing a 'both sides of the argument'. The second issue here is the length of the question. Which of it is not required? As you can see, the revised draft is much shorter, without being ambiguous. We want questions to be as concise as possible, so try to use simple sentence structures which are accessible to all.

Looking at this second example then, what feedback would you offer the writer? As this question covers a number of distinct elements, we felt it would be helpful as students to provide an image drawing attention to the key points. The decision was taken to include an image of the high jumper, clearly showing the movement and highlighting the joints referred to in the question. This also ensures all students are provided with some knowledge of the sport in case it was less familiar to them.

Finally, take a look at the synoptic 15-mark question for A-level PE. What feedback would you offer the writer? While no question is perfect, this question was approved and published in this form. The standard bullet pointing to make the different athletes clear, the command 'Analyse' starts the question after the stem, and the word 'each' was important, so students needed to address each of the athletes individually. The synoptic element is highlighted, as while the main question is on dietary supplements on manipulation, the requirement to refer to energy systems throughout the answer is clearly stated.

As well as writing questions, we are also required to produce marks schemes to assess students' responses. Mark schemes have a vital role to play in the validity and reliability of an assessment. If mark schemes are not clear, then two examiners may award different marks for the same response. This prevents the assessment from being reliable.

The first type of mask scheme we use are points-based mark schemes. And as previously mentioned, this type of mark scheme is most commonly used for short answer questions. Points-based mark schemes are mark schemes where marks are given for correct points made by a student. Correct answers are clearly defined, and each correct answer directly corresponds to a mark. Some reasons why validity can be lost in PE mark schemes include, a mismatch between the question in the market scheme, a lack of transparency in the marking rules. In this adapted example from GCSE PE, the question asks students to explain two short-term effects of exercise, but the mark scheme awards marks for simply stating 'short-term effects'. The command word in the question was not consistent with the type of answer credited in the mark scheme. It's possible that students could write much more than they needed to because of the mismatch between the question and the mark scheme. This will take more time than needed, possibly increasing the chance of them only mentioning a single effect.

The second example shows why underlining has been used to make market rules more transparent. In this example, the terms 'skillful', 'well learnt', and 'adaptable' have been underlined to make it clear that reference to multiple programs in isolation is not creditworthy. Obliques are also used to show how alternative answers can be credited, but the overall point can only be awarded once.

The second type of mark scheme we're looking at today is a level of response mark scheme, Level of response mark schemes are broken down into levels, each of which has a descriptor. The descriptor for the level shows the average performance of the level. Indicative content is also provided beneath the level of response grid, support, and guidance only. This type of mark scheme is better suited to unconstrained questions, i.e., those where the students have more freedom in how they respond. In PE, we typically use them for extended response questions, as it allows the examiners to mark students' responses holistically, and to credit a wider range of responses. Remember, these questions are more open and allow students to answer them in a number of different ways. Consequently, the indicative content in a mark scheme will not be definitive.

When using level of response mark schemes, examiners apply a principle of best fit, first, identifying the appropriate level, and then the mark within the level. The focus of the level of response mark schemes is on assessing the quality of students' responses, rather than assigning a mark per correct point, as would be the case with points-based marks claims. It's useful to share mark schemes with students so that they can judge the level of their own work. Also, by understanding how that work is assessed, they can understand how to improve their answers and exam timing.

Hopefully, this session has given you an insight into the assessment design process at AQA, and also how we try to maximise the validity of our assessments. In addition, I hope you've picked up some techniques which you can apply in your classroom to help improve student performance. Thank you for taking the time to watch.