In Defence of Progress 8 (Part 3)

In Defence of Progress 8 (Part 3)

Problem 3: What Does Progress 8 Mean?

(It is too sensitive to individual students who disproportionately affect the whole school figure)

(Wordle created from August 24th DfE guidance to new exam gradings – https://www.gov.uk/government/news/guide-to-gcse-results-for-england-2017 )

Many teachers just don’t understand what the Progress 8 number means – it’s a little opaque. This is understandable. Here’s a quick summary which I hope helps.

Let’s imagine you teach in a school with 200 year 11 students, and on average, all of your cohort should average a grade 5 if they were in the 50th percentile school. They are taking 8 GCSEs, with English and maths counting twice, so we’ll call it 10 GCSEs. This is what Progress 8 is based on.

It isn’t based on targets. It simply compares how each student did compared to all the other students in the country who had the same starting point.

 

How the Numbers Work

To get a progress 8 figure of 0.0 in this example, the total grades of your year 11 would be 200 students x grade 5 x 10 subjects = 10,000 school points.

How vulnerable is this figure to underperforming students? Let’s suppose that 10 of those students take no GCSEs at all, and get no grades, nil points.

Your school progress 8 score would now be 190 students x grade 5 x 10 subjects = 9500 school points.

Compared to the national expectation of 10,000.

Divided by the number of students in your cohort.

9500 – 10000 = -500 / 10,000 = -0.05

 

Is Progress 8 Skewed by a Few Students?

So, having 10 whole students in your cohort gaining not a single GCSE at all would have a very small effect on your Progress 8 figure. When we look at it like this, Progress 8 doesn’t look too vulnerable to variations in your cohort at all.

Let’s get more extreme. If there were 20 students getting nothing, this would be 10% of the cohort. We would expect this to mean that they would get 10% worse Progress 8. This would be -0.1

This is in fact what the calculation gives: 180 student x grade 5 x 10 subjects = 9000 school points, compared to 10,000 expected points.

-1,000 / 10,000 = -0.1

So, actually, we see that the figures are not that sensitive to chance. Having 20 students sit no GCSEs whatsoever, all of whom were expected to get the average grades in the school, would have a small effect on progress 8. And this effect would be the same as 40 students in the year group getting only 5 gcse grades, instead of 10. That’s a massive number who would have to go wrong to shift your P8 by 0.1

However, if all 200 students get one grade less, then the same is true, our school would score -0.1.