Strictly Come Scoring

By Kate Kelly
Published 15 Oct 2015

Kate Kelly explains how low scores in the populist TV dance competition encourage fairness

It's definitely autumn: the leaves are turning, the mornings are darker and the Pumpkin Spice Latte has returned to Starbucks. For me, this can only mean one thing: a new series of Strictly Come Dancing!

This might seem like an odd topic for a CERP blog post, but all competitions have some form of assessment at their heart. As much as I love the glitter, the dancing and the dodgy puns, I'm also fascinated by the parallels between Strictly and the work we do in CERP.

One such parallel comes from the scoring system. For those not in the know, this is how it works:

each of the four judges gives each dance a mark out of 10, leading to a total score out of 40 once all the contestants have danced, the public can vote for their favourite each contestant is assigned two scores based on their rank positions: one based on the judge's scores, one based on the public vote. In each case, the highest ranked contestant is given a score equal to the number of couples competing, and so on in descending order these are added together to form an overall score the two contestants with the lowest combined score are entered into the dance-off.

So whether or not a contestant goes into the dance-off is half determined by their scores from the judges, and half by the public vote. In the same way, a GCSE or A-level candidate's final grade is usually based on two or more assessments, each of which contributes a certain percentage of the overall marks. We could, therefore, think of each week of Strictly as a dance exam comprising two components, each contributing 50% to the final outcome. This contribution is known as an intended weighting. But whether an intended weighting is achieved or not depends on the extent to which each component discriminates between the contestants.

Let's illustrate this with an extreme example. Imagine if every contestant was given the same score by the judges. In this scenario, all contestants have the same place in the judges' rank order: the judges have not discriminated at all between the contestants. As can be seen in the table below, when you combine these rankings with the rank from the public vote, each contestant's place in the final rank order is determined entirely by the public vote. (The voting data is not published, so the public vote rank order here is entirely fictional.) By failing to discriminate between the contestants, the judges' scores don't contribute at all to the final outcomes an actual weighting of 0%. By the same token, if every couple had the same number of votes, their place in the final rank order would be entirely influenced by the judges' scores.

Obviously such an extreme scenario is unlikely to occur, but it neatly demonstrates how the variation in each set of rank orders can affect how much they contribute to the overall outcome. It's for this reason (among others) that good discrimination is one of the features we look for in our assessments, and we routinely monitor the achieved weightings of our examinations.

It's also for this reason that Craig is my favourite judge. Known for being Strictly's Mr Nasty, Craig has a reputation for being the harshest judge, regularly dishing out twos and threes where the others give fours and fives. But while Len, Darcey and Bruno might be nicer, by rarely giving marks below four they aren't distinguishing between dancers as clearly. Craig's willingness to give out lower scores increases the range of total scores, discriminating better between contestants. This in turn helps to ensure that the judges' scores make a contribution to the overall outcomes.

So if you've ever complained that Strictly is nothing more than a popularity contest, don't blame Craig!

Kate Kelly

Photo courtesy of BBC/Guy Levy. Strictly Come Dancing is on BBC One

For further reading, see Strictly statistics, published in the Feb 2017 edition of Significance, which explores these issues and more in greater detail

Keywords

• Fairness