The art of moderation

Series one
Bonus: Episode four | 3 June 2019

In this Inside Exams bonus episode, Sandra Allan, AQA’s Head of Curriculum for creative arts, and Robin Drummond, one of AQA’s assessment design managers, talk through how they ensure art is marked reliably and accurately.

Featured in this podcast

Sandra Allan – Head of Curriculum for creative arts at AQA

Robin Drummond – Assessment Design Manager for performing arts at AQA

Transcript

Sandra: Hi my name is Sandra Allan. I am the Head of Curriculum for creative arts.

Robin: Hello, my name is Robin Drummond, I am the Assessment Design Manager for various performing arts subjects.

Sandra: As the Head of Curriculum, my role is to ensure AQA is the go-to board for the arts. I am here to support teachers across all creative subjects.

A lot of the time I receive calls from teachers asking me, “What will I tell a student, or how do I convince my senior leaders why they should do art?" I would always start by saying, “Art is by no means an easy subject, as many people like to think. Nor is it a very complicated one if you approach it right. You need to strike a good balance between doing the things you love and backing them up with the hard evidence and research. Gone are the days where you just need to draw lifelike portraits and pretty landscapes to get awarded marks for your efforts and your natural ability to be creative. You must be inclined to analyse what you see and experiment with various styles and medium if you want to fulfil the conceptual requirements of this curriculum.”

However, this might add to the appeal when it comes to someone who isn’t naturally gifted, in their own admission. It can also lead to fulfilling artistic career opportunities. Let me give you an example, art textiles, the students love it. Three quarters of a million people within the fashion and textiles industry work within the UK. This is more than the motor industry. I am not saying students don’t study mechanical engineering etc, it is a huge part of our economy, but there are jobs out there waiting for art students.

Art is a good starter qualification. You can gain skills, knowledge and creativity. It’s the top skill being sort for by LinkedIn and the world economic forum place it third in their rankings of the most important skills to offer, for entering the work place – just below, digital knowhow and complex problem solving. Really, what I always say is schools need to actively encourage students to take open-ended subjects where they can develop concepts and make them become a reality.

Aside from the obvious though, there are many advantages to taking art and this is what students need to think about and need to understand, but also senior leaders need to understand how it’s going to help the students, in other areas of their study. For instance, improved focus. Taking the time to look at art pieces in a gallery or to create a piece of artwork of your own, requires a great amount of focus and concentration. Learning how to switch off from your surroundings and apply yourself fully to a task, is an important skill that can be transferred to other situations in life whereby you need to give something, or someone, your full undivided attention. Another example, and this one is my favourite, is the enhanced, personal confidence. Art is all about self-expression. By displaying artworks in the public you are going out there seeking accreditation. It opens us up to scrutiny and judgement. That is why some of the most famous artists are seen as controversial and are sometimes encouraged to stand up and explain their works and defend themselves from widespread media criticism.

Your confidence, communication skills and expression, are distinguished, developed and improved upon and can then be used as a preparation for other areas of life. It’s also a great social setting. The art community is made up of so many varied, creative people – you can go from graphic designers, photographers, printmakers, sculptors, fashion designers.

And finally, a better appreciation of history, geography and culture. Sources of art date back centuries and have been evident consistently throughout history. The earliest cave paintings, for example, gave us an idea of what life was like during the ages and also suggested where colonies of humans and animals might have taken up residence. Moreover, looking at artwork from different parts of the world can help people understand the different countries, their heritage and their traditions.

Another questions are there academic art benefits? Yes. Even the most professional of businesses are in need of some artistic flair. If a career is what your students have their hearts set on, then you already know that creativity and imagination are the two most basic skills you need to succeed, and an art course can help you develop these skills in abundance.

When it comes to careers, as I’ve said before, an art and design course can be beneficial and take you on routes including fine art, illustration, teaching of humanities, fashion design, even down to jobs in a tattoo parlour. However, if you’re students are not completely sure on the area of the field that they want to pursue, you have opportunities to choose among a wide range of sixth forms and further studies and then you can decide at a later date. But, as I mentioned, even serious businesses like banks and solicitors go on the search for artistic talent, particularly when it comes to advertising. A lot of organisations, museums and galleries have competitions where students have the opportunity to exhibit their work. These are really worth looking into and it’s a really good way for students to look into where they would like to take their career in art.

Robin: Okay so, as an assessment design manger, my main focus is on how we make judgements about the quality of students’ performances in various subjects and that’s in terms of fairness and reliability. That’s really important and at the heart of what AQA does.

When it comes to art and design, the best place to start with is the assessment objectives. So, all qualifications have assessment objectives and they’re actually set by Ofqual and they’ll be the same for all art and design courses across all exam boards. The assessment objectives for GCSE and A-level are very, very similar, but they do vary slightly to reflect the increased demand at A-level. I will just talk about the similarities between GCSEs and A-levels. So, there are four assessment objectives for each. When it comes to actually marking pieces of work in art and design, there is a very simple approach taken at both A-level and GCSE. Each component at A-level and GCSE and AS obviously, is assessed in exactly the same way.

Each piece of work, or each component, is awarded a mark out of 96 and that mark is given initially by the teacher in the school, and the 96 marks are spread evenly over the four assessment objectives. So, because with art and design the work is marked by teachers in centres, it is really important that AQA takes steps to maintain the standard across centres, nationally. Actually, this is one of the requirements Ofqual makes of exam boards that we put into place rigorous, moderation procedures, to ensure this.

So, in terms of what we actually do at AQA, we send a visiting moderator into centres to remark a sample of student’s work in that particular centre. How this works in practise is that a visiting moderator will contact the school and arrange a time to visit the school, and remark a sample of the student’s work. This sample can range in terms of number, depending on the number of students in the centre, but it’s always designed to represent all of the students taking art and design in that centre. So the top mark and the bottom  mark will be taken and an even spread of marks within that range, to represent the students in that centre as accurately as possible. Once the moderator has remarked the sample of work in a centre, they then compare their marks to the teacher’s marks, taking into consideration a small tolerance and making a decision about whether further work needs to be looked at, or whether the marks that they’ve awarded can be sent to AQA.

A tolerance is used because awarding marks entails human judgement, so when a moderator looks at a centres’ marking, it’s unlikely that there will be precise agreement. So, two equally well trained markers can give the same piece of work slightly different marks. And, in fact, the same marker may give the same piece of work different marks, at different points in time. So there is a small number of marks, a small tolerance, that allows for that difference and that’s perfectly fine.

Once the moderator has finished marking the sample of work, they’ll then complete what’s called a centre feedback form which contains various information that’s given to the centre, including the quality of marking. So the school gets some direct feedback on exactly how well they marked, based on the moderator’s marking of the sample.

One of the questions that we often have with regards to moderation is, “What happens when a centre hasn’t applied the standard and their marks are outside the tolerance of the moderator’s marks?" In these cases, then AQA will apply an adjustment to the centre’s marks to bring the standard back into line with AQA standards. The reasons why we do this can be varied. So, it can include new inexperienced teachers marking for the first time, a new specification, possibly a lack of attendance at teacher standardisation events, or, possibly even, poor, internal standardisation, within centres.

So, if a centres’ marks need to be adjusted the regression approach is applied. The regression compares two sets of data. The centre marks and the moderator marks, in order to find the relationship between the two. The relationship is represented by a line of regression or a line of best-fit. This line is then used to recalculate every student’s mark. It is very important to treat all students the same. Therefore, all students, whether or not they were in a sample, have their marks recalculated, based on the regression line. This also helps to maintain the rank order of candidates as determined by the centre, which research shows, is usually accurate, even if the marking is generally lenient or harsh. For each candidate, the line gives an estimate of the mark that the moderator would have given. The estimate is based on the sample of work that was actually inspected by the moderator.

No centre likes to have their marks adjusted, but hopefully it will only happen once to a centre after which they can implement the advice given by the moderator, and avoid it in subsequent years. If you are in a centre that has had marks adjusted, one of the things that we’d highly recommend is that you attend one of AQA’s face-to-face standardisation events. These are free and they are based on live students’ work and it’s a great chance to really get in line with AQA standard and practise your marking and get some really good feedback from AQA.

It would be easy to think that a subject like art and design was trickier to assess. Or, the way it was assessed was looser or vaguer than more objective subjects. But, hopefully, you’ve seen that that isn’t the case with AQA art and design, both at GCSE and A-level. The way we approach assessing these subjects is very rigorous and in line with all of our other qualifications.