Mission impossible: naming rainbow colours

Series one
Episode seven | 8 July 2019

How are mark schemes written? What are the different types of mark schemes? And how can you make use of mark schemes in the classroom? Craig Barton meets AQA Assessment Design Manager Suzanne Oates to find out what examiners consider when they write mark schemes, and history teacher Carmel Bones shares the fun classroom games she plays to familiarise her students with them.

Featured in this podcast

Craig Barton – Maths teacher, podcaster and author

Laura – Drama teacher

Suzanne Oates – Assessment Design Manager at AQA

Carmel Bones – History teacher

Episode resources

The anatomy of a mark scheme

Transcript

Craig Barton: Hello and welcome to Inside Exams. I’m Craig Barton, I’m a maths teacher with 15 years of classroom experience. I can make good use of visualisers and mini white boards but exam materials can feel a bit foreign. So across this series I am uncovering the secrets Exam Boards might usually keep to themselves and asking how they can help us become even better teachers. Today, I want to talk about mark schemes.

Now I’ll tell you what I’m thinking about as my point of reference here, pub quizzes. It all starts off perfectly jolly until you’re asked to swap papers with the team next to you. I don’t know about you, but I spend more time sneaking a look at the person who’s marking my answers, making sure they’re being fair and defending why my answer is right, thank you very much. And over on my table I might be more inclined to be lenient in ticking an answer if it’s my [make] sheet. I know that his brain was on the right track and would have got there in the end.

Humans aren’t always the most reliable markers, so what role does a solid mark scheme play in ensuring examiners award marks to our students impartially? Before I head off to find out, I need to know what else you need to know.

Laura: Hello, my name is Laura and I’m a drama teacher. The mark scheme for each of the drama components relies on subjective interpretation, an example being the assessment bands for the C2 devising log which are split into four sections; excellent, good, reasonable and limited. What further support can you put in place for teachers to ensure we’re getting consistency?

Craig Barton: I wonder too. As a teacher does your perspective of your students’ potential impact how leniently or harshly you mark their work? How can we write mark schemes that prevent that from happening? I’m often to meet Suzanne Oates, one of AQA’s Assessment Design Managers, to pick her brains.

Well Suzanne it is fantastic to meet you. Throughout this series I’ve been learning loads about how exams are put together but one missing piece so far has been the mark schemes themselves, which obviously are a fundamentally important part of the exam process. So I am thinking you are just the person to help me out with this and I’m going to open up with a two mark question for you.

Suzanne Oates: Okay.

Craig Barton: What are mark schemes and why do we need them?

Suzanne Oates: So mark schemes are the method by which we categorise students into bands in order that we can grade them and give them an appropriate grade. And we need them because we sometimes have hundreds of examiners marking responses from different students. We have obviously thousands of students taking our exams and we need to make sure that all of the examiners who are applying the mark scheme can do so in a consistent way. So we can’t just let them give what marks they feel, we have to give them some structure for actually applying those marks.

Craig Barton: Who actually writes the mark scheme? Is it the people writing the questions themselves?

Suzanne Oates: Yeah, that’s right. So we have a number of subject experts who are involved in the creation of papers. And when a lead assessment writer, is what we call them, is producing the question papers, they also produce the mark schemes. And in fact we actually recommend that the mark schemes are written before the questions are written.

Craig Barton: Oh wow, really?

Suzanne Oates: Which sounds a little bit counterintuitive but one of the key issues that we find is somebody will write a question, they think they know what they’re asking and then they write the mark scheme and those two things don’t necessarily match up. Whereas if you actually write the mark scheme first, if you can define what we call the outcome space, so that’s the kinds of answers that you’re expecting to see, what are you wanting to elicit? The question almost from then writes itself, because you’ve expressed what you want to see in the answers and then you think “Okay, what question would elicit this?”

Craig Barton: Wow.

Suzanne Oates: Now it doesn’t always happen like that but that is what we recommend. And there are lots of things that can be avoided if you do the mark scheme first. So one of the key examples is hurdles in a mark scheme. So a question might be “Discuss the differences between cats and dogs”, I mean it probably wouldn’t be but let’s take that as an example. And then if you’ve written a mark scheme that to get the top marks, you say “Students must have discussed at least three differences between cats and dogs”. Well you haven’t told the students that and it could be that a student only talks about two differences but they do it in real depth and if what you’re looking for is breadth, you need to specify that in the question. So if you can articulate what you want to see in the outcome, then you know what the question should be.

Craig Barton: Let me get into the mind of the examiner here. Is it literally the case that they are essentially answering that question in a few different ways and then almost kind of marking themselves as a way to then create the mark scheme? Or is it a bit more complicated than that?

Suzanne Oates: In a sense, so what we’ll usually use for an essay based question, let’s say the cats and dogs as an essay question. We would usually use what we call a level of response mark scheme. So what we do with those is we actually define different levels of outcomes. So we explain the kinds of things we would expect to see in a weak response, a sort of slightly better response, a middling response, a pretty strong, good response and then an excellent response. And you can have a different number of mark schemes depending on the tariff of the question.

So rather than actually sort of writing an actual response, you’re thinking about the features that you’re looking for. And that can be very challenging to do because you need to remember the assessment objectives, so you need to make sure that if it’s a knowledge question or an understanding question, that’s what the mark scheme targets. If you’re requiring an answer to analyse the source, that’s what you’re accrediting.

So you’ve always got to remember what’s the assessment objective? What’s the command word of the question, have we asked for a discursive response or an explanation? Have we asked for key points to be laid out or do we want to do some comparison? Bear that in mind and actually then start to define what features we’d expect to see in each of the levels.

So if you’re looking at a basic response then you’re probably not seeing much comparison happening, you’re probably just looking at a statement of a small number of facts, there’s limited analysis of those facts. But then when you get up to the top band, you’re seeing a wider range of points, you’re seeing some analysis going on and it’s separating and differentiating those levels that’s the key thing.

Craig Barton: Okay. So I’ve my head round level of response. Is there any other technical lingo I need to know about different types of mark scheme?

Suzanne Oates: Yes. So we have objective mark schemes. These are usually in response to a multiple choice or a very low tariff question and there is just a right answer.

Craig Barton: I like those ones.

Suzanne Oates: There is simply a right answer. And we can get a high degree of accuracy with the marking of those because it’s either right or wrong. And then we also have penny point mark schemes so this is where, usually for fairly low tariff questions, although it can get quite high, but they have a range of correct answers.

Craig Barton: Right.

Suzanne Oates: So the key to success with these is to make sure that the questions are really tightly defined so that a student can’t get marks just sort of by writing almost anything, they’ve got to be pretty tightly defined. And the indicative content in the mark scheme must be very clear and very clear is we’re going to be very clear how to apply it. So an example could be “State three colours that appear in the rainbow”.

Craig Barton: Okay.

Suzanne Oates: And then the mark scheme would be red, orange, yellow, green, blue, indigo and violet. And you could also define what’s not acceptable, so you could say, for example, purple is not enough.

Craig Barton: Got it. Now I’ll tell you something that keeps coming up throughout this series, these flipping assessment objectives, they must be the bane of examiners lives right.

Suzanne Oates: Yes, exactly.

Craig Barton: So just talk to me about how do they impact how mark schemes are put together and how do they kind of constrain in a way what examiners can do?

Suzanne Oates: So assessment objects basically define what type of question we can ask. And so, for example, one of the subjects I look after is computer science and if we’re asking students to look at an algorithm and do a trace on that algorithm, so actually work out what the algorithm is doing and what the outputs would be, that would be a AO2 apply question.

Craig Barton: Right.

Suzanne Oates: If we’re asking them to explain what that algorithm does, that could be either maybe AO2 analysis or it could be understanding, it just depends how the question is structured.

Craig Barton: Flipping heck.

Suzanne Oates: So you’ve got to be really careful because you have to have a balance because the assessment objectives, they’re defined by our regulator Ofqual.

Craig Barton: Yes, yes.

Suzanne Oates: In terms of how much coverage you’ve got to have.

Craig Barton: They keep coming up, Ofqual.

Suzanne Oates: Yeah. And they’ll say “Okay, well these papers have to be 30% AO1, 30% AO2, what have you. So we’ve got to make sure that the percentage of questions in the final paper match the assessment objectives that we start off with. So we have big tracking grids and all the assessment writers know that they need to target, in particular proportion with each of the assessment objectives.

Craig Barton: Got it, fantastic. So if we go back to the evolution of these mark schemes. So I’m the examiner, I’ve got my cats and dogs question going on here. I’ve banged out a few kind of sample responses and I think I’ve put my mark scheme together. Who sees it next?

Suzanne Oates: So it’s goes through quite a long process. There will be a reviser who comes and looks at it who is some fresh eyes on it. And then a series of people, teachers and subject experts and me as an Assessment Design Manager, who will be looking to see, to make sure that the content in the mark scheme matches the question, that’s one of the key things that I ask.

And then there are a bunch of things that we will look for. You mentioned maths earlier, are we giving marks for working out even if they’ve got the answer wrong? So how are those ones? So even relatively straightforward mark schemes, or what you think might be straightforward, there’s quite a lot of thought goes into them because you’ve got to know exactly what you’re going to credit. Do they only get the marks if they get the answer right or is there, you know, can they get marks along the way?

We have to make sure we’re not penalising the same mistake twice, so if they make an error, say if we went back to the dogs question and they said there was a particular difference between breeds of dogs and that sort of flowed through their answer but the kind of ideas behind it were sound, you have to make decisions about how far you’re going to penalise one mistake, if it’s a fundamental misunderstanding of something in a subject, then that’s fine.

But if you’re just penalising it for its own sake, because you know, they’ve made this mistake. And that’s particularly true in calculation questions, if they’re carrying a mistake through, you want to try and avoid that and credit the logic and the understanding that’s behind it rather than just saying “Oh they get no marks because they made one mistake at the beginning”.

Craig Barton: Say the worse things to mark are those kind of three and four mark questions where the kid makes an error, line one, their answer is not right and you think “Oh, oh, I can’t just cross this because I’ve got to look for these follow on marks”. Or they have a technical term I assume, I call them follow up marks. Is there a technical term?

Suzanne Oates: We usually say follow through.

Craig Barton: Oh perfect, I’m almost there, I’m almost there. Now the problem with that of course is that there’s almost an infinite number of paths that that answer can take, right, depending on the mistake the kid has made. And if they make it line one and it’s a four or five mark question, it’s sprouting off all over the show and no mark scheme in the world can kind of cover all those different things. So how do mark schemes deal with that kind of stuff?

Suzanne Oates: We’ll give some instructions, so they’ll be, “If the student”, say the first step should be multiplying something by 8,000 and they’ve done it by 800, we kind of know that sometimes happens.

Craig Barton: Yeah, of course.

Suzanne Oates: We’ll know often because we’ve got, you know, teachers involved in this process, we know the kinds of mistakes that students tend to make. So we’ll give some instructions about if they’ve done this bit incorrectly, just concern yourself with the logic of the next bit. And then we’ll usually give a couple of examples, common ones we expect, fully worked out with the marks allocated so you can see, okay well they haven’t got the first mark point but the second step they’ve made in the calculation, even though it’s got the wrong values in it, is actually correct. So we can give that mark point. They’ve got the third step wrong, they’ve gone astray something but actually by the time they’ve got to the fourth step, they are doing the calculation we’d expect, so they could get maybe two out of the five.

Craig Barton: Yeah. What fascinates me about this is that’s maths and maths is a relatively objective subject, you’re either right or wrong. And yet still within mark schemes in mathematics I find there’s a certain degree of subjectivity. Have they meant to do that? Can I take it from their line of working that they are displaying the correct method and so on? That’s mathematics and we refer enough things in maths, but what on earth do you do in a subject like English or history? There must be so much subjectivity that comes in that again, I can’t imagine how a mark scheme would account for all that.

Suzanne Oates: I think it goes back to again clear description of the levels, so what are the key features? And one of the thing that we, you know, we’re learning and developing all the time and I spend a lot of time talking to my examiners about is, it isn’t good enough just to say “Bad, getting a bit better, well this is alright, oh this is quite good and brilliant”, it’s not good enough. You need to explain what the features of that are, what does excellent analysis look like? What kind of things are you looking for?

And another key point is also indicative content, so the kinds of things that we expect to see in answers and sometimes, you know, you have an examiner who is an expect in a particular field and they write this kind of dissertation lens, really highly complex indicative content and there’s no 16 or 18 year old in the country that’s going to be putting that in their answer. So you’re kind of cutting that back and saying “What are the key points? What are the kinds of things kids are going to be talking about?”

And then the other key thing that we use is standardisation material, so you can’t really underestimate seeing the mark scheme being applied. So before we start standardising these teams of examiners, a bunch of senior examiners get together, choose some exemplar scripts and they will exemplify, sometimes a really clear example of a particular mark.

But also sometimes the most useful ones are ones where there is a little bit of a debate and discussion. You can actually tease out some of those finer points and say “Well, we ultimately put this answer in band three, even though it’s showing some features of band four. It’s solidly in band three, it’s near the top of band three because of those, you know, band four features, but it’s not doing enough to get into the next one”.

And so if you can have some of those discussions, either face to face or we have online standardisation as well, that really is key to embedding the understanding of that mark scheme in examiners’ minds. So clearly differentiated levels, they’ve got to be clearly different and then having lots of really good exemplar materials so that you can train examiners.

Craig Barton: So I can see now how the mark scheme may evolve before the actual exam is sat, but correct me if I’m wrong, and this is down to my limited experience of maths marking, is it true sometimes the mark scheme changes after the kids have actually taken the exam?

Suzanne Oates: Yeah, that’s right. We do try and keep it to a minimum. We’re getting much better. Every exam series we learn more, we understand more, we know what to expect more and we try and build that in upfront. But we are surprised sometimes, we’re surprised by how questions are interpreted, you know, it’s been through every single possible check you could think of and yet a large proportion of students have interpreted a question differently from how you intended it to be taken. So we have to be flexible at that point and if we’re seeing some unexpected responses, then we need to be able to credit those.

What we try to do, and certainly one thing that we do in computer science, is we might put accept mark points in. So it’s not 100% right, we would rather, when teachers are looking at those mark schemes after the results are out, when they’re made available, that what they’re teaching is the core content of the mark scheme but there are some points. And you know, over time you’re trying to sort of help teachers get the knowledge that they need and kind of embed that there.

So we will try and do that upfront but during the standardisation process or when we’re looking at scripts, if we see that students are interpreting a question in a different way than we intended, then we will make some changes. But we are trying to minimise that as much as possible to make sure we’re doing it right upfront and we’re not just kind of adding in anything. And it’s not, you know, it’s not if one student has done something different then we’ll go “Oh, put this in as well”. But if we’re seeing evidence that we should be making amendments then we do.

Craig Barton: Flipping heck, the thought process that goes into this is absolutely fascinating. So Suzanne it’s been an absolute pleasure to speak to you today. Thank you so much.

Suzanne Oates: Thank you.

Craig Barton: Okay. It’s clear that an incredible amount of work goes into creating mark schemes but how can we as teachers ensure we’re making the most of them? Could they actually be useful teaching tools that have been under our noses this whole time? I’m off to meet Carmel Bones, she’s a long serving history teacher and she’s also an educational consultant working with teachers and students. So I can only imagine the wealth of experience she’ll have to share with us.

So we have heard from Suzanne about all the thought that goes into mark schemes but what I’m interested in now is what does that look like in the classroom. So I’ve come all the way up to sunny Carlisle to speak to Carmel who’s going to hopefully enlighten me.

Carmel Bones: Oh hello Craig and welcome to Cumbria.

Craig Barton: Thank you very much. I’m going to open up with a deep question, we might as well get the ball rolling in style here. How do you feel about mark schemes in general?

Carmel Bones: Well that’s a biggie. Obviously they are I suppose a necessary evil is how I would describe them. In terms of using them in the classroom, I have kind of mixed views in that it depends on how and when you introduce them to the students, there isn’t a one size fits all situation. In terms of how I feel about them, then obviously teachers have got to be au fait with them, they’ve got to understand the rubrics and what’s expected. We want our students to be successful at the end of the day.

In terms of how I feel about them with use of students, I think they should be introduced to them pretty early on but it very much depends. You don’t want to bore the pants off the students, you don’t want to bog them down with the rubrics and it depends on their starting points. If you’re working with a high flying group of Year 13s, for example, it might be something that you introduce them to very, very early on actually because they may as well see what they’re going to be up against. And they don’t want to be frightened of it and you don’t want to keep it a secret. They like to be kind of on the inside if you like.

Having said that, other students, it could switch them off, it could disengage them and history is my background and I firmly believe you need to enthuse students about the subject, the content, the knowledge. And in some cases the mark scheme and being able to meet with the criteria kind of takes care of itself. So it really does depend, judicious use of mark schemes is what I would suggest.

Craig Barton: Let’s look at some practical ways of using it. I want to start by how do mark schemes, if they do, how do they inform your planning?

Carmel Bones: Well I think that’s a really good point and I think they should. I sort of take the backward planning approach and quite often with, you know, my team of historians we would sit down and look at the end game first and think well where do the students need to be? What are they up against? And let’s think how we can kind of drip feed or take a staggered approach to introducing and releasing the information to the students.

Quite often as well having a choice available, so saying to the students, “These things are all available on the website, so you can have a look”, and you might have some real keen students who will go ahead and use them. Again, you know, think about post 16 students particularly, they might use their mobiles, fire away, look things up and be ahead of the game. So choice I think is very important as well.

In terms of use in lessons, you’re going to laugh now, but I think my approach would be to kind of get fun with mark schemes. Now I know this is a contradiction in terms, but the idea of let me use that in a moniker, the idea of making the students familiar with the mark schemes first, there’s the first bit.

Craig Barton: Okay.

Carmel Bones: Getting to grips with how to use them, okay.

Craig Barton: Oh I see where we’re going, yes.

Carmel Bones: And then now apply them, there’s your fun, you see.

Craig Barton: Nice, I like it, I like it.

Carmel Bones: It’s fun, what’s not to like? And again that would really vary on how you did it depending on the students. But I can give you a few examples if you like.

Craig Barton: Go for it. Because I’ll tell you why, because I use mark schemes a lot in maths and they are the opposite of fun, the way I’m using them, so I am desperate for some help here, so go for it Carmel.

Carmel Bones: Well I might be clutching at straws but I’m trying. So first of all you’ve really got to sell it to the students. You know, I’ve done all sorts of things where it’s play the examiner style, so you’ll have the music on, Mission Impossible, you know, “De, de, de-de-de”, cue the music. So you’re selling it to them that they are going to be the examiners and I’ve even said “Oh guys, you know AQA have got a conundrum, they’ve got these scripts and they’re a bit stuck and they really don’t know how to mark them”.

And once the kids are pretty much sort of brought inside the exams in a way, they love that challenge and they’re like “Whoa, is it real Miss? Let’s have a look”. And you then get scripts from wherever, it can be from the Exam Board website, it can be, which I’ve done quite a lot, a teacher with a parallel group, and you know, you anonymise things. And what I’ve found is if you keep them authentic, don’t bother typing them up, goodness me, teaching is hard enough, don’t give yourself another job. Use the students’ real hand, anonymise them and give the students a pack of mark schemes.

Again, the more sort of ‘missionify’, you know the more ‘gamify’ you can do the better. So perhaps have the mark schemes in an envelope, in a sealed envelope on the table when they come in, different coloured paper, card, anything to just add a bit of sort of spice to the lessons. Have the students then maybe in teams of, I would say, two or three, maybe four at the most, any more than that you’ll have passengers who sit back and don’t take part. So you have a tight kind of marking team, if you like, and they then apply the mark schemes, and in my experience, they’re really, really good at it.

They look at the mark schemes, they look at the indicative content and they apply it to the scripts that they’re given and then they put them in a pecking order and they rank them. And then, you know, you would feed back to the students and reveal what the marks might actually be. If you haven’t said it was a stuck situation, depending on how you’ve sold it to them. Or you can say “That’s great, I’ll feedback once I’ve heard from AQA” and you can keep them dangling to the next lesson or whatever, have another go, revisit it. It’s something that I think you’ve got to do periodically rather than save it up for one time or keep it all to the end.

Also things like chopping up the mark schemes and getting them, you know, maybe they should sort of pre-date what I’ve just described, would be part of the F of fun, familiarisation. Chopping up the mark schemes and getting them to reorder them, sort them into the different levels, just literally giving them a mixed bag of slips and saying “Well guys, can you make sense of these? We need to photocopy them quickly”, timer on again, Mission Impossible, you know, “You’ve got three minutes to get these into order”. And then when you reveal that they’ve done it and they’ve achieved the task, they get such a great sense of accomplishment.

Craig Barton: Yes.

Carmel Bones: And competency is very motivational and if they get it right and they’re introduced to it in that way, I know you’ll be laughing now Craig, but they might say something like, you know, “Can we do the mark schemes again? Can we have another look at some other examples?” And you know –

Craig Barton: My kids have never said that so this is music to my ears this.

Carmel Bones: – I’ve been teaching a long time [laughter]. So it’s things like, for example, them saying “Oh you know, can we have”, I mean back in the day, you know, “Can we have a look at some Cs against Ds?” Which I guess, you know, would maybe be fours as against five, and really unpick and be precise, what’s missing? What do you need? What do you not need? You know, that kind of approach.

Also what’s really interesting and what I love is, I mean the students are pretty harsh in their application of them.

Craig Barton: Yeah, yeah, I can imagine.

Carmel Bones: They have very high expectations but you’ll hear them saying “Oh they’ve just waffled there, they haven’t answered the question, come on, answer the question”. So it makes them very focused in terms of allowing themselves to think about what to miss out as well and not to waffle. And I sort of say “Come on guys, you know, these guys are under pressure, it’s in the exam, give them a break”.

And they sort of say “They should know better, look at the command words, look at the date frame, they’ve gone outside of the date, that’s waffle then. That question said what was important about the arrival of Mary Queen of Scots into England, not her execution and the aftermath, it’s the arrival, come on guys”. And they really do become very, very empowered. So I’ve found it quite useful in that regard.

Craig Barton: So you mentioned here that it’s something that you drip feed throughout the year because I think hearing this, a mistake I’ve made with mark schemes is I just introduce them like a month before the exam, whenever we start doing the practice papers with Year 11, that’s when I go mark scheme mental with the kids. Is that a mistake? Let’s say you’ve got a Year 11 class, when would they, be first be exposed to one of those activities?

Carmel Bones: Well Year 11 is too late in my view.

Craig Barton: Oh okay.

Carmel Bones: GCSE is lost and won in Year 10.

Craig Barton: Right, okay.

Carmel Bones: I’d have revealed them, to some groups again, it just depends, very, very much depends, I would have introduced some familiarisation certainly in Year 10. And the use of them, possibly play the examiner type game, maybe even having things like laminated frames. I’m a big fan of laminated frames whereby if you think of an old fashioned kind of overhead projector, transparency, something laminated that the students overlay onto a piece of work.

And then they would apply the mark scheme to their own work initially perhaps and/or to a peer, somebody else in their class. And then of course use of visualisers as well to put students work then under the visualiser and show everybody well they’ve got the mark scheme and they’re equipped to make comments and annotate the work that way.

Craig Barton: Now one thing I’ve done in maths and I’m intrigued whether this works in history, is we’ll take a question and I’ll say to the kids first off, “How many marks do you think this question is worth?” And secondly “Can you create your own mark scheme for the question”. Would that work in a subjective subject like history or is that a bit too much?

Carmel Bones: No, it absolutely would. But I think if we go back to the fun with mark schemes, it would come back to the you. They’ve been familiar with them, so they’ve had some practice of looking at them and looking at them in terms of their own work before they then go onto use them in that way. So you might give them, you know, almost you know, a WAGOLL, what a good one looks like.

Craig Barton: What was that, a WAGOLL?

Carmel Bones: A WAGOLL, yeah. What a good one looks like –

Craig Barton: God, I don’t know if I like this, okay, yeah.

Carmel Bones: – so you would give them perhaps a mark scheme and a piece of work that’s been levelled and marked with some examiner’s comments, i.e, yours perhaps, or one of your colleagues. And then you say to them “Right, let’s look at another example and you now apply this”. So they’ve gone beyond the familiarity stage, they know the levels, they know some of the indicative content and I would replicate that with a similar question. So you know, you might be like “How convincing is source X about Y?” Well we would practice that and then we would give different content. So the application is the same but the content is different.

So you know, how convincing is this source about Francis Drake’s motives for going on a voyage of discovery? We might then swap that and say, you know, “How convincing is this source thinking about the issues Elizabeth I faced in terms of her marriage?” You know, so you would use the same rubrics but you would then alter the content. And once you got the students familiar with that and in the groove of it, maybe with some low tariff questions at first, you would then move onto more complex mark schemes in the 16 markers.

Craig Barton: 16?

Carmel Bones: Exactly. It’s like jumping off a cliff isn’t it?

Craig Barton: Wow.

Carmel Bones: And I would say, you know, show things sooner rather than later if appropriate. Build it up throughout Year 10 so by the time you come to Year 11 they’re very au fait with this and they’re spotting gaps in each other’s work as well.

Craig Barton: Wow. One thing we do as a Maths Department is in our departmental meetings, we’ll put up a question from a real life student or, as you say, we’ll get one off the AQA website or wherever it is, and we will try and mark it ourselves and it will be maybe a three mark question or a four mark question and inevitably we won’t agree on how many marks that question would get.

Now that is a maths question which, as you say, you’d imagine is kind of black and white. I cannot even begin to think what that would be like with a 16 mark history epic.

Carmel Bones: Yeah, yeah.

Craig Barton: So do you talk to colleagues about mark schemes and how to allocate marks? And do those, I can imagine they could get quite heated.

Carmel Bones: Very much so, oh yeah. I mean you know, it’s been going dark some nights when we’ve been still having a ding dong about how much something is worth. But again it just comes down to practice really. We’ve done that in departmental meetings numerous times and had tricky questions. There are a few ways round it. I’ve encouraged my teaching team to mark for Exam Boards as well so they get an insight into exams and I have done GCSE and A’ Level. So I would, probably in my department, be the final arbiter as it was, because I’d had that experience.

But it is a learning curve and I remember when I first started applying the mark schemes as an examiner, I was a bit too much as a teacher. I remember one of my Team Leaders saying to me “Why have you rewarded that person with that mark?” This was an online task. And I said “Well I could see where they were going”. And he said “Stop, they haven’t done it. you have to just reward what is there. You’re now thinking you know, person X, oh they’ve done their best, you’ve got to move into examiner head rather than teacher head”.

So I encourage my teaching team to try and get into examiner head and look at what was there. And we did all kinds of things like we swapped each other’s scripts around to some extent as well.

Craig Barton: Yes, that’s a good one.

Carmel Bones: And you know, I mean there’s pros and cons because you don’t want someone to be too harsh and, you know, we all, I talked about relationships and the importance of that. But it can work where you look at just something bold and bold and apply the mark scheme to the letter of the mark scheme because that is ultimately what will happen.

Craig Barton: Another thing I’ve found that I wonder if you’ve found this Carmel, is sometimes when I mark students’ work using a mark scheme, it identifies big holes in my teaching, kind of things that I’ve missed out and that’s dead, dead useful for me because obviously I need to rectify that. Is that something that you’ve found and is that a kind of almost a hidden benefit of using mark schemes, the fact that it really forces you as a teacher to zone in on any potential gaps in your teaching, if that makes sense?

Carmel Bones: Absolutely. And we talked about the idea of teaching backwards earlier on.

Craig Barton: Yes.

Carmel Bones: I think the teacher should completely familiarise themselves with the mark scheme before they issue anything to the students. And something that I’ve found really good practice throughout my career, was always, always to do the question myself.

Craig Barton: Yes, yes, yes.

Carmel: And so often I’m quite surprised by, and I know people are in a rush and all the rest of it and they issue stuff to students and they haven’t really thought it through sometimes themselves.

Craig Barton: Hmm, I’ve been there, yeah.

Carmel Bones: So I think you doing it yourself, and when you realise where the gaps could be, try and plug the gaps before you then set the exam because you [brought] the question, because you want the outcomes to be as high as they possibly can. So what I’ve then done, when I’ve found that situation, is gone back into lessons, given the whole class feedback but said “Right, on this table guys, we’re going to have like a free workshop on that particular question. If you scored highly on it, crack on, you don’t need to be with me having this extra bit”.

Also finding experts in the room, you might have a class of 32, it’s too hard for you to do it all, pick one or two experts who scored highly on a question and then they can have a little, a team of their classmates around them and help them as well.

Craig Barton: Yes. Final question, we’re probably going to have plenty of new teachers listening to this podcast, looking for some insights and advice. If we’ve got a less experienced teacher listening, who wants to use some of these techniques and wants to really get their head into mark schemes and turn them into a teaching tool not just like an add on for assessment, have you any advice for them?

Carmel Bones: I would say use the idea of backward planning, so look at the mark schemes first, see where your students are headed, use those to inform your teaching and your sequence of lessons. Use things like, you know, play the examiner, have these things ready to activate as and when is appropriate. Again approaches like getting students to write their own mark schemes, mark work from other classes. I mean all kinds of different ways.

And also a new teacher or a less experienced teacher, don’t be afraid to ask for help, you know, your Head of Department is not a scary person, they were you once, you know, maybe not too long ago, so go to them and don’t be afraid, it’s not a sign of weakness at all.

Craig Barton: Amazing stuff. Well Carmel, I’ve absolutely loved every minute of this conversation. I have learnt so much and you’ve helped me see how I can put the fun back into mark schemes which is fantastic, so thank you so much for your time today.

Carmel Bones: Oh, thank you Craig, it’s been an absolute pleasure and thanks for your kind words.

Craig Barton: I can’t quite believe how much enthusiasm Carmel has for mark schemes but I have to say I think it’s rubbed off on me. I might just go away and come up with some of my own Mission Impossible style marking games. If you’d like to visualise the sorts of thought processes Suzanne was telling me about earlier, head to the podcast show notes where you’ll find some annotated mark schemes to work through in your own time.

Next time I’ll be chatting with yet more exam writers, markers and pioneering teachers, so if you want to swot up throughout exam season, make sure you rate, review and subscribe to the podcast.

Until next time, goodbye.