Science

This video focuses on how assessment principles apply in GCSE Science, looking at the assessment framework for Science, how questions are structured and how the demand of a question can be changed to suit its purpose.

Transcript

Hello, I'm Gary Topham. I'm the chief examiner for GCSE Chemistry. Today, I'm going to be talking to you about assessment principles in GCSE Science papers. The objectives of today's presentation are to understand the purpose of summative assessments in science to discuss what a good science question looks like, to understand the regulatory framework of GCSE science examinations, to see examples of how questions are asked and improved, and the next steps and how you can use these ideas to support students in your schools.

A summative assessment should assess the right things in the right way to produce accurate and useful results. Our lead assessment writers aim to produce papers that are accessible. The language, context and question type need to be such that the only barrier to a student being able to answer the question is their knowledge and understanding of the science. We need to avoid idiom and jargon that can confuse students and keep context as simple as possible. They need to be fit for purpose. They need to make sure that the assessment actually does address the topic or the specification reference that it's designed to do. It needs to differentiate between students of different skill levels.

Questions are set at different levels of demand to ensure that students of different abilities can demonstrate what they know and can do. In the GCSE sciences the extended response questions are very good at differentiating students. They follow very important principles when setting assessments. They need to be reliable to always give the same or broadly similar results each time it's taken. They need to be valid so you can trust the outcomes of the assessments.

The trust is dependent upon the reliability of the results and the other principles we will be discussing today. They need to be set at the correct level of demand, designed appropriately to be accessible to students of all abilities. In GCSE science this is partly addressed by tiering, but also by ramping demand of questions within each paper. They need to produce useful results and this is summative assessment for learning. How do you use results to help students to improve in schools and GCSE exam papers allow us to order a group of hundreds of thousands of students in terms of science ability.

A question that does these things can be considered a good science question. This example shows how we put these assessment principles into practice to structure a good GCSE Science question in terms of accessibility, differentiation and fitness for purpose. It is a standard demand Biology question, but these principles apply across all the GCSE sciences. The question starts with an overarching statement which every part of the question will link to. Often this stem is this question's about, but we do mix it up a bit as you will see in our exam papers. The Stem helps accessibility by giving the students a focus for what the question is about, and if we change the stem, we can increase or decrease the demand, thus helping differentiation. I'll talk about how we can change the demand of questions a bit more later.

The question itself always puts the command word at the start of the question line so that it is clear to students what we want them to do. We only use one command word in a sentence, keeping the language straightforward and helping with accessibility. The sentence containing the command word is usually the last sentence before the answer lines so that what we're asking the student to do isn't lost in any other information given and helping them keep focus. We have a set of common words that we use consistently, and this is published on the website, along with what we mean by them. So teachers and students can understand what is needed to answer each one. If there are any necessary prompts, such as the numbers of responses we require, we always put a prompt in bold so it is clear to the student what we want them to do. Again, ensuring accessibility. It also helps with fit for purpose, as it is then clear that, for example, we want two things. We always give the number of marks available for the question. Again, giving students a prompt as to the amount they should be writing in their answer. The number of answer lines we give for a question, usually two per mark, although some questions, only one word or very short phrase is needed. So we'll only give one line again, clues students into how much we're expecting them to write. In this example, we have numbered the answer lines to emphasize that we are looking for two answers.

However, if a student wrote both answers on one line, they would still be marked. The numbers are simply there as a prompt. If we're looking at increasing the demand for the question we might remove the numbers on the answer lines. The second part of this question has a structure very similar to the first part. This part has a stem that is related only to this question. As you can see, there is more information for this part which is designed to provide the student with guidance on what they are going to be required to do. In this question, the guidance tells the story of how the lipids in milk are broken down, which is intended to help students focus on the breakdown products and which one of these is acidic, the clue being caused the ph to decrease. We keep sentence length to less than 20 words if possible, to keep the demand of reading appropriate for GCSE students. Sentences may be split up if necessary to ensure that they fit the rules to ensure appropriate reading age. Again, the actual question is the last thing the student reads before the answer lines, and it is a clear command word at the beginning.

The question is developed along with its mark scheme, to ensure that what we are asking the student to do in the question is actually what we are expecting in the mark scheme. That includes checking that it is addressing the specification references and the assessments objectives we are assigning to it to ensure the question is fit for purpose. As an example, if the command word is describe, the marks scheme should not be expecting an explanation. OFQUAL passed the content and the structure of each specification prior to the publication of new specifications ready for first teaching in 2016. I'm going to talk about the structure of all GCSE science papers and the constraints that are imposed which ensure that exam papers each year are consistent from one year to the next.

The structure is the same for the GCSE separate science papers, except for maths content and the proportions are the same for all the GCSE combined Science trilogy and Synergy papers. This is the blueprint for a GCSE trilogy Chemistry Foundation paper. Every section must be completed for every paper to ensure consistency from one exam series to the next. These are criteria that were agreed with OFQUAL so must be met. There is no wiggle room. This consists of the specification reference.

There's maths marks and references. These should be a minimum of 10% maths on biology papers, 20% on chemistry papers and 30% on physics papers. Maths references can be found at the end of each specification. Working scientifically marks and references. These are found at the start of every specification. Required practical marks. There should be a minimum 15% of the marks from the context of the required practicals in each specification. This usually means that there are 15% of the marks on each paper from the required practicals, 15 marks and a separate science paper, ten or 11 marks on a trilogy paper and 15 marks on a synergy paper. Assessment objective descriptors for each assessment objective can be found in the specification. On all GCSE science examination papers, there should be 40% A01, 40% A02 and 20% A03 with small tolerances to ensure that the papers work. Each AO is broken down further and there is a requirement that all AOs are covered across each specification. Grades on a foundation tier paper, there should be 60% of the marks at grades 1 to 3 and 40% of the marks at grade 4 to 5. I will carry on outlining what is required for each paper that is written by looking at a blank higher tier grid from the same specification.

This is the blueprint for the higher tier paper. The higher tier grid contains all the same headings as the foundation tier, with the only difference being that there are three grades that need to be fulfilled. There should be 40% of the marks at 4 to 5, 40% at grade 6 to 7 and 20% at grades 8 to 9. This is one of the reasons why entries are very important on tiered papers, as a grade four student on an F tier paper will be able to answer a significant amount of the paper, whereas the same student will often only answer a small proportion of a higher tier paper. Other features that are highlighted are common questions. These are all graded at 4 to 5 and are common to both foundation and higher tier papers so that direct comparisons can be made during the awarding process. Open or closed questions, there are a maximum of 32 closed marks, multiple choice linking lines and gap filled questions on a foundation tier paper and 14 on a higher tier paper. It is rare that this number is reached as often this type of question lowers the demand of a question. Extended Response Questions. There should be a minimum of 10% of the marks on a foundation tier paper as extended response. 15% on a higher tier paper. This could be both written and multi-step calculations, which are worth four or more marks. The last columns are for internal use, identifying the type of marker for each item. Multiple choice questions are auto marked where the responses are keyed in and marked by a computer. General mark questions are fill in the gaps or linking line questions that require no expert knowledge. The majority of each paper is expert marked where subject specialists are used in the marking process. I have tried to give you a feel for the complexities of putting the exam paper together and now I want to take you through the process of writing an examination paper and explain how seemingly the same question can assess science at different grades. Now we know what a question paper looks like. I want to give an idea of the whole paper production process. It takes about 15 months to produce a paper that is fit for purpose. The responsibility of producing the paper is the lead assessment writer's. However, they are supported by a team of experienced teachers and examiners.

The idea is to fine tune the questions in a paper to ensure the paper provides the information required. So how do we do that? Often the same basic question can be used at different levels of demand by changing the format. In this example, which is set at low demand and so is suitable for a foundation tier paper, we have given the equation for kinetic energy for the students to use. This keeps the demand of the question low. By not including the equation the same question will be more demanding and will be used to test students working at a higher level of skill. Often as what we call standard demand targeted roughly at grades 4 to 5, we would ask students to write down the equation before we asked the calculation question. The demand of a question can also be lowered by asking it in a slightly different format. This question, which appeared in a 2021 trilogy higher tier paper, could be eased in demand by adding the prompt, you should answer in terms of electrons.

The prompt clues students more into what they should be thinking and writing about so makes it more accessible to students of lower ability. The ,ark scheme would be almost identical however, the prompt means that the first part of the mark scheme is more accessible, electrons are delocalised. Whole questions can be revised to ensure that they are as accessible as possible and elicit the outcomes desired. Questions are designed to test students science knowledge, not their interpretive skills. This is the start of an exam question that would still be relevant for 4.10.3.2 on the current GCSE chemistry paper 2. So as you read through the question, it appears to be okay. All the parts fit with the beginning of the question. So there is something of a flow about the items in the question.

There are issues, however, with the order of the questions and the contents of some of the questions. These are the sorts of discussions that will be happening during the paper production process, as well as ensuring that the paper meets the criteria discussed earlier. I have had to squeeze this question in a bit so that it fits onto the slide. How would this question look different if it was on a more recent examination paper? I must reiterate that these revisions are done by a paper production team on what I think is required could be different from what other members of the team think. Often there are two or three different opinions on the same item. A consensus is reached to try to make the question as good and clear for the student sitting the exam as possible. The start should be a hanging stem. A short introduction to the question that links the whole question. In this case, as simple as this question is about steel, the bar chart is not actually used for the first question. So I would move that first question to above the bar chart. Then there comes a question of format. We do not write draw a ring around the correct answer questions anymore. This would most likely be a multiple choice question. A sentence completion questions should have the words at the end of a sentence. A further stem,  steel is a mixture of iron and other elements is added. Elements would have to be used because carbon is not a metal, so it could not be iron and other metals. Multiple choice questions need to be direct questions. So what name is given to a mixture of iron and other elements? The options are usually given in alphabetical order.

Graphs, diagrams, etc. are given figure numbers, in bold and referred to in the question. It is now appropriate to give the bar chart. The numbering system for questions is now different. There are only numbers, no letters and roman numerals for sub parts of items. So now 1.2 can be asked. The command word plot is given at the start of the question line. 1.3 the information given is the same, but the instruction line is split to help students with what is expected from them. The command word suggest is now at the start of the question. Then prompt use Figure one then comes after. There would be white space between the parts of this item as well as answer lines but I have omitted them to reduce the number of slides.

There would also be the mark tariff on the right hand side below each item. 1.4 the original question in the next item has a very difficult question structure. It would need amending to a stainless steel contains 0.2% carbon 8% nickel 18% chromium and iron. The options from the original question could then be used with tick boxes similar to 1.1. Part two of the original question then asks about the steel with 0.2% carbon, which was the carbon content of the stainless steel in the previous question. Therefore, one of the other steels would have to be used from the bar chart with a different percentage of carbon added.

A standard format is now used for sentence completion questions. The space for the answer will always be at the end of the question and there is standard wording. Complete the sentence. Choose the answer from the box. The question works as it is. The answer line should not be on a line of its own, so some words are moved on to the next line. I hope that you can now understand some of the processes that exam questions have to go through before they can be used to assess students. The first and in some ways the most important idea is do not reinvent the wheel.

For the AQA examinations, Exam Pro is an excellent resource. However, use with care when Exam Pro is used, ensure that only content that is required is tested. Assigning AOs requires a little more experience. On the most recent exam papers AOs are identified for individual parts of each question. Providing questions at the correct grade is also important as questions often have a mix of grades. Are they suitable for what you are trying to assess? Try to employ ideas of different proportions of marks at different grades and AOs. The AOs are not available on Exam Pro at present. However, Exam Pro is being redeveloped to allow searching using AO. Watch this space. I hope you found this presentation informative and useful and I hope you can use it to be able to provide your students with the ideas to help them improve further.

Thanks for watching.

Questions you may want to think about

• How can you use these insights to prepare your learners for exams?
• Do your internal assessments reflect the approach of the exam? To what extent do you want them to?
• What’s the most important or surprising thing that you’ve learned? How might it influence your teaching?