The problem of reverse excellence gaps

This post compares the performance of primary schools that record significant proportions of disadvantaged high attainers.

spiral-77493_1280It explores the nature of excellence gaps, which I have previously defined as:

‘The difference between the percentage of disadvantaged learners who reach a specified age- or stage-related threshold of high achievement – or who secure the requisite progress between two such thresholds – and the percentage of all other eligible learners that do so.’

It draws particular attention to the incidence at school level of sizeable reverse excellence gaps where disadvantaged learners out-perform their more advantaged peers.

According to my theoretical model reverse gaps threaten equilibrium and should be corrected without depressing the achievement of disadvantaged high attainers.

In this post:

  • The measure of disadvantage is eligibility for the pupil premium – those eligible for free school meals at any time in the last six years (‘ever 6 FSM’) and children in care.
  • The measure of high attainment is Level 5 or above in KS2 reading, writing and maths combined.

.

National figures

The 2014 Primary School Performance Tables show that 24% of the cohort attending state-funded primary schools achieved KS2 Level 5 or above in reading, writing and maths combined. In 2013, the comparable figure was 21% and in 2012 it was 20%.

In 2014 some 650 primary schools managed a success rate of 50% or higher for the entire cohort, up from 425 in 2013 and 380 in 2012

The comparable national percentages for disadvantaged learners are 12% in 2014, 10% in 2013 and 9% in 2012. For all other learners (ie non-disadvantaged) they are 24% in 2012, 26% in 2013 and 29% in 2014.

In 2014, there were 97 state-funded schools where 50% or more of disadvantaged learners achieved this benchmark, compared with only 38 in 2013 and 42 in 2012. This group of schools provides the sample for this analysis.

Chart 1 below illustrates the national excellence gaps over time while Chart 2 compares the proportion of schools achieving 50% or higher on this measure with all learners and disadvantaged learners respectively.

.

REG graph 1

Chart 1: Percentage of disadvantaged and other learners achieving L5+ in KS2 reading, writing and maths, 2012-14

Chart 1 shows that all rates are improving, but the rate of improvement is slower for disadvantaged learners. So the socio-economic achievement gap at L5+ in reading, writing and maths combined has grown from 15% in 2012, to 16% in 2013 and then to 17% in 2014.

REG graph 2 

Chart 2: Number of schools where 50% of all/disadvantaged learners achieved L5+ in KS2 reading, writing and maths, 2012-14

Chart 2 shows steady improvement in the number of schools achieving outstandingly on this measure for all learners and disadvantaged learners alike (though there was a slight blip in 2013 in respect of the latter).

Since 2012, the proportion of schools achieving this benchmark with disadvantaged learners has increased more substantially than the proportion doing so with all learners. At first sight this is a positive trend.

However Chart 1 suggests that, even with the pupil premium, the national excellence gap between higher-attaining advantaged and disadvantaged learners is increasing steadily. This is a negative trend.

It might suggest either that high-attaining disadvantaged learners are not benefiting sufficiently from the premium, or that interventions targeted towards them are ineffective in closing gaps. Or perhaps both of these factors are in play.

 

Schools achieving high success rates with disadvantaged learners

The 97 schools achieving a success rate of 50% or more with their disadvantaged high attainers are geographically dispersed across all regions, although a very high proportion (40%) is located in London and over half are in London and the South-East.

.

Reg graph 3

Chart 3: Distribution of schools in sample by region

 .

Nineteen London boroughs are represented but eight of the 97 schools are located in a single borough – Greenwich – with a further five in Kensington and Chelsea. The reasons for this clustering are unclear, though it would suggest a degree of common practice.

Almost half of the sample consists of church schools, fairly equally divided between Church of England and Roman Catholic institutions. Seven of the 97 are academy converters, six are controlled, 42 are aided and the remainder are community schools.

Other variables include:

  • The average size of the KS2 cohort eligible for assessment is about 40 learners, with a range from 14 to 134.
  • The percentage of high attainers varies from 7% to 64%, compared with an average of 25% for all state-funded schools. More than one quarter of these schools record 40% or more high attainers.
  • The percentage of middle attainers ranges between 38% and 78%, compared with an average of 58% for state funded schools.
  • The percentage of low attainers lies between 0% and 38%, compared with the national average for state-funded schools of 18%. Only 15 of the sample record a percentage higher than this national average.
  • The percentage of disadvantaged learners ranges from 4% to 77%, compared with the national average for state-funded schools of 31%. Roughly one in five of the sample has 50% or more, while almost two in five have 20% or less.
  • The number of disadvantaged pupils in the cohort is between 6 and 48. (Schools with fewer than 5 in the cohort have their results suppressed). In only 22 of the sample is the number of disadvantaged pupils higher than 10.
  • In 12 of the schools there are no EAL pupils in the cohort but a further 11 are at 60% or higher, compared with an average for state-funded schools of 18%.

Overall there is significant variation between these schools.

.

School-level performance

The vast majority of the schools in the sample are strong performers overall on the L5 reading, writing and maths measure. All but five lie above the 2014 national average of 24% for state-funded schools and almost half are at 50% or higher.

The average point score ranges from 34.7 to 27.9, compared with the state-funded average of 28.7. All but 15 of the sample record an APS of 30 or higher. The average grade per pupil is 4B in one case only and 4A in fourteen more. Otherwise it is 5C or higher.

Many of these schools are also strong performers in KS2 L6 tests, though these results are not disaggregated for advantaged and disadvantaged learners.

More than four out of five are above the average 9% success rate for L6 maths in state-funded primary schools and almost two out of five are at 20% or higher.

As for L6 grammar, punctuation and spelling (GPS), some two-thirds are above the success rate of 4% for all state-funded primary schools and almost two out of five are at 10% or higher.

When it comes to the core measure used in this analysis, those at the top of the range appear at first sight to have performed outstandingly in 2014.

Four schools come in at over 80%, though none has a disadvantaged cohort larger than eight pupils. These are:

Not far behind them is Tollgate Primary School, Newham (71%) but Tollgate also has a cohort of 34 disadvantaged learners, almost three times the size of any of its nearest rivals.

What stands out from the data above all else is the fact that very few schools show any capacity to replicate this level of performance over two or three years in succession.

In some cases results for earlier years are suppressed because five or fewer disadvantaged pupils constituted the cohort. Leaving those aside, just 6 schools in the sample managed a success rate of 50% or higher in 2013 as well (so for two successive years) and no school managed it for three years in a row.

The schools closest to achieving this are:

  • Tollgate Primary School, Newham (71% in 2014, 50% in 2013 and 40% in 2013)

Only 9 of the sample achieved a success rate of 30% or higher for three years in a row.

The size and direction of excellence gaps

Another conspicuous finding is that several of these schools display sizeable reverse excellence gaps, where the performance of disadvantaged learners far exceeds that of their more advantaged peers.

Their success rates for all other pupils at L5 in reading, writing and maths combined vary enormously, ranging between 91% and 10%. Nineteen of the sample (20%) is at or below the national average rate for state-funded schools.

But in a clear majority of the sample the success rate for all other pupils is lower than it is for disadvantaged pupils.

The biggest reverse excellence gap is recorded by St John’s Church of England Primary School in Cheltenham, Gloucestershire, where the success rate for disadvantaged learners is 67%, compared with 19% for other learners, giving a huge disparity of 48 percentage points!

Several other schools record reverse gaps of 30 points or more, many of them church schools. This raises the intriguing possibility that the ethos and approach in such schools may be relatively more conducive to disadvantaged high attainers, although small numbers are undoubtedly a factor in some schools.

The ‘cliff-edge’ nature of the distinction between disadvantaged and other learners may also be a factor.

If schools have a relatively high proportion of comparatively disadvantaged learners ineligible for the pupil premium they may depress the results for the majority, especially if their particular needs are not being addressed.

At the other extreme, several schools perform creditably with their disadvantaged learners while also demonstrating large standard excellence gaps.

Some of the worst offenders are the schools celebrated above for achieving consistency over a three year period:

  • Fox Primary School has a 2014 excellence gap of 34 points (57% disadvantaged versus 91% advantaged)
  • Nelson Mandela School a similar gap of 28 points (54% disadvantaged versus 82% advantaged).

Only Tollgate School bucks this trend with a standard excellence gap of just two percentage points.

The chart below illustrates the variance in excellence gaps across the sample. Sizeable reverse gaps clearly predominate.

 .

REG graph 4

Chart 4: Incidence of reverse and normal excellence gaps in the sample

Out of the entire sample, only 17 schools returned success rates for advantaged and other learners that were within five percentage points of each other. Less than one-third of the sample falls within a variance of plus or minus 10%.

These extreme variations may in some cases be associated with big disparities in the sizes of the two groups: if disadvantaged high attainers are in single figures, differences can hinge on the performance of just one or two learners. But this does not apply in all cases. As noted above, the underperformance of relatively disadvantaged learners may also be a factor in the reverse gaps scenario.

Ofsted inspection reports

I was curious to see whether schools with sizeable excellence gaps – whether normal or reverse – had received comment on this point from Ofsted.

Of the schools within the sample, just one – Shrewsbury Cathedral Catholic Primary School – has been rated inadequate in its last inspection report. The inspection was undertaken in July 2014, so will not have reflected a huge reverse excellence gap of 38 percentage points in the 2014 KS2 assessments.

The underachievement of the most able is identified as a contributory factor in the special measures judgement but the report comments thus on the achievement of disadvantaged learners:

‘Although in Year 6, pupils eligible for additional government funding (the pupil premium) reach similar levels to their classmates in reading, writing and mathematics, eligible pupils attain lower standards than those reached by their classmates, in Years 2, 3 and 4. The gap between the attainment of eligible and non-eligible pupils in these year groups is widening in reading, writing and mathematics. In mathematics, in Year 3, eligible pupils are over a year behind their classmates.’

Two further schools in the sample were judged by Ofsted to require improvement, both in 2013 – St Matthew’s in Surbiton and St Stephen’s in Godstone, Surrey. All others that have been inspected were deemed outstanding or good.

At St Matthew’s inspectors commented on the achievement of disadvantaged learners:

‘Weaknesses in the attainment of Year 6 pupils supported by pupil premium funding were identified in 2012 and the school took action to reduce the gap in attainment between this group of pupils and their peers. This gap reduced in 2013 so that they were just over one term behind the others in English and mathematics, but there is still a substantial gap for similar pupils in Year 2, with almost a year’s gap evident in 2013. Support is now in place to tackle this.’

In 2014, the KS2 cohort at St Matthew’s achieved a 53% success rate on L5 reading, writing and maths, with disadvantaged learners at 50%, not too far behind.

At St Stephen’s inspectors said of disadvantaged learners:

‘The school successfully closes the gap between the attainment of pupils who benefit from the pupil premium and others. Indeed, in national tests at the end of Year 6 in 2012, the very small number of eligible pupils was attaining about a term ahead of their classmates in English and mathematics. Focused support is being given to eligible pupils in the current year to help all fulfil their potential.’

A more recent report in 2015 notes:

‘The school is successfully closing the gaps between disadvantaged pupils and others. In 2014, at the end of Key Stage 2, disadvantaged pupils outperformed other pupils nationally and in the school by about three terms in mathematics. They also outperformed other pupils nationally by about two terms nationally and in the school in reading and writing. Disadvantaged pupils across the school typically make faster progress than other pupils in reading, writing and mathematics.’

It is not clear whether inspectors regard this as a positive outcome.

Unfortunately, Tollgate, Nelson Mandela and Fox – all three outstanding – have not been inspected since 2008/2009. One wonders whether the significant excellence gaps at the latter might impact on their overall inspection grade.

.

Pupil Premium allocations 

I was equally curious to see what the websites for these three schools recorded about their use of the pupil premium.

Schools are required to publish details of how they spend the pupil premium and the effect this has on the attainment of the learners who attract it.

Ofsted has recently reported that only about one-third of non-selective secondary schools make appropriate use of the pupil premium to support their disadvantaged most able learners – and there is little reason to suppose that most primary schools are any more successful in this respect.

But are these three schools any different?

  • Fox Primary School has pupil premium income of £54.7K in 2014-15. It explains in its statement:

‘Beyond all of this, Fox directs a comparatively large proportion of budget to staffing to ensure small group teaching can target pupils of all attainment to attain and achieve higher than national expectations. Disadvantaged pupils who are attaining above the expected level are also benefitting from small group learning, including core subject lessons with class sizes up to 20. The impact of this approach can be seen in the APS and value added scores of disadvantaged pupils for the last 2 years at both KS1 and KS2. The improved staffing ratios are not included in pupil premium spend.’

  • Nelson Mandela School has so far not uploaded details for 2014-15. In 2013-14 it received pupil premium of £205.2K. The statement contains no explicit reference to high-attaining disadvantaged learners.
  • Tollgate Primary School received pupil premium of £302.2K in 2014-15. Its report covers this and the previous year. In 2013-14 there are entries for:

‘Aim Higher, challenging more able FSM pupils’ (Y6)

In 2014-15 funding is allocated to pay for five intervention teachers, whose role is described as:

‘Small group teaching for higher ability. Intervention programmes for FSM’.

.

Conclusion

The national excellence gap between disadvantaged and other learners achieving KS2 L5 in all of reading, writing and maths is growing, despite the pupil premium. The reasons for this require investigation and resolution.

Ofsted’s commitment to give the issue additional scrutiny will be helpful but may not be sufficient to turn this situation around. Other options should be considered.

The evidence suggests that schools’ capacity to sustain Level 5+ performance across reading, writing and maths for relatively large proportions of their disadvantaged learners is limited. High levels of performance are rarely maintained for two or three years in succession.

Where high success rates are achieved, more often than not this results in a significant reverse excellence gap.

Such reverse gaps may be affected by the small number of disadvantaged learners within some schools’ cohorts but there may also be evidence to suggest that several schools are succeeding with their disadvantaged high achievers at the expense of those from relatively more advantaged backgrounds.

Further investigation is necessary to establish the association between this trend and a ‘cliff-edge’ definition of disadvantage.

Such an outcome is not optimal or desirable and should be addressed quickly, though without depressing the performance of disadvantaged high achievers.

A handful of strong performers, including the majority of those that are relatively more consistent year-on-year, do well despite continuing to demonstrate sizeable standard excellence gaps.

Here the advantaged do outstandingly well and the disadvantaged do significantly worse, but still significantly better than in many other schools.

This outcome is not optimal either.

There are very few schools that perform consistently highly on this measure, for advantaged and disadvantaged high attainers alike.

Newham’s Tollgate Primary School is perhaps the nearest to exemplary practice. It receives significant pupil premium income and, in 2014-15, has invested in five intervention staff whose role is partially to provide small group teaching that benefits high attainers from disadvantaged backgrounds.

Fox Primary School has also acted to reduce group sizes, but it remains to be seen whether this will help to eliminate the large positive excellence gap apparent in 2014.

This is a model that others might replicate, provided their pupil premium income is substantial enough to underwrite the cost, but the necessary conditions for success are not yet clear and further research is necessary to establish and disseminate them.

Alternative approaches will be necessary for schools with small numbers of disadvantaged learners and a correspondingly small pupil premium budget.

The Education Endowment Fund (EEF) is the obvious source of funding. It should be much more explicitly focused on excellence gaps than it has been to date.

GP

May 2015

Advertisements

3 thoughts on “The problem of reverse excellence gaps

  1. Hi GP,

    This is a really interesting analysis – I had not thought about “reverse excellence gaps” before. You’re right to be wary of year-on-year comparisons, as variations across primary cohorts can distort these comparisons, particularly when cohort sizes are small. I also like how you acknowledge the problems with a “cliff-edge” for pupil premium eligibility – clearly there needs to be more graduated approach to funding for disadvantaged pupils and schools.

    My question concerns pupil mobility. I wonder if the schools that demonstrate reverse excellence gaps are ones where more able PP pupils join from lower performing schools, perhaps after a house-move to be close enough to a desired secondary school? So the reverse excellence gaps happen because of pupil sorting due to ambitious parents (who have PP children). Are the schools you highlight over-subcribed?

    I also wonder about variation in characteristics across PP pupils. PP only captures financial status, nothing further. We know that the recession led more pupils to be eligible for PP, who may have not been in that financial position a few years earlier. Perhaps these “temporary” PP pupils are the ones who are doing well? Ethnicity may also play a part. Perhaps a non-White British PP pupil is different too? It’s possible that detailed characteristics like these mean the PP attainment gap isn’t easily comparable across schools.

    Thanks again for sharing this piece of work.
    Meena

  2. Hi Meena

    Thanks for your comments and positive feedback.

    I definitely agree that the characteristics of the pupil premium learners could play a part in this, or at least those that are associated with higher attainment, but I wonder whether the more significant issue is that the schools concerned are so heavily focused on their pupil premium beneficiaries that they are comparatively neglecting those who don’t attract it. They are probably satisfied if those children get their L4s but, for whatever reason, they are pushing the pupil premium cohort a little further.

    If that’s the case though, it’s surprising as I can’t put my finger on any incentives in the system that would encourage such behaviour – rather the reverse in fact. Increased Ofsted scrutiny could be a factor in future, but it’s unlikely to have impacted on the 2014 SATs.

    I definitely think it’s an area that would benefit from closer study, but that would require access to the full NPD for the schools in question.

    GP

  3. Really interesting to read this. There are probably as many reasons for these reverse excellence gaps as there are schools where they occur. Meena has certainly highlighted some possibilities. Because you qualify for pp by being eligible for FSM in the last 6 years, it can catch students whose families have gone through temporary hardship or family breakdown. I am regularly surprised by which of my students qualify and also by those who don’t. Maybe there is a better way to select students for extra funding, perhaps giving schools more of a say.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s