Strictly Come Scoring (part II)
By Kate Kelly
Published 22 Dec 2015
Strictly Come Dancing is beloved for both its tangos and tantrums. Shock exits are something of a tradition for the BBC One series (see: Pixie Lott, 2014) but Anita Rani's departure ignited outrage among fans, which was expressed everywhere from Twitter to the Telegraph. Despite consistently impressive performances over the last 12 weeks, Anita lost out to the elegant-but-erratic Katie Derham, who joined Jay McGuiness, Georgia May Foote and Kellie Bright in the finals.
Anita's fans claim that Katie's progression must have been based on something other than dancing ability the inexplicable charm of Anton Du Beke, perhaps? given that Anita has generally been a better dancer. However, based on my experience as an assessment researcher, I suspect that exits like this are an inevitable feature of the Strictly assessment model.
In my last blog, I outlined how the two least successful dancers were identified via a combination of the judges' scores and the public vote. The final decision as to who stays and who goes, however, lies in the judges' views following a 'dance off'. This means that a brilliant dancer, caught on a bad day, can be knocked out in favour of a generally poorer performer.
The same might be said about exams in general. A GCSE or A-level grade is usually based on just a few performances, meaning it can only ever be a snapshot of a candidate's ability, even if it is an accurate reflection of their performance on that assessment. Is this unfair? In some respects, yes it is. But that's not to say that taking earlier performances into account is any fairer.
Imagine, if you can, a world in which Jeremy Vine has progressed from dad-dancer extraordinaire to a Strictly superstar whose jive rivals Jay's. If Jay and Jeremy were in the dance off, who should win? They have very different past performances, but both could be considered worthy of reward: Jay has been consistently excellent throughout, but Jeremy would have shown stratospheric improvement. Should we value Jay's consistency more or less than Jeremy's improvement? Which would be the fairest outcome?
In the same way, pupils might take very different routes to achieve the same level of performance. The end-of-year examination might feel unfair when a pupil who is generally very able has an 'off day'. But an alternative approach that considers performance over the whole course would give greater rewards to the pupils who were consistent than to the pupils who made huge improvement, even if the ultimate attainment is the same.
In this respect, what feels like a simple question of fairness becomes a question about what we value, and what should be rewarded. This is a more complex question, to which there can be many valid answers. Both Strictly and end-of-year examinations take the approach that the final performance should be what counts, not the journey that has been taken to get there. While this means that, sometimes, the wrong dancer wins the dance off, it also means that students and dancers alike are not penalised by their earlier performances.
So, I sympathise with the BBC fairness seems like such a simple concept in principle, but the reality is much more nuanced.
Photo courtesy of BBC/Guy Levy
For further reading, see Strictly statistics, published in the Feb 2017 edition of Significance, which explores these issues and more in greater detail
About our blog
Discover more about the work our researchers are doing to help improve and develop our assessments, expertise and resources.