Warning: teachers - don't get seduced by the data

Self-confessed data junkie and headteacher Michael Dix says teachers need to view data with a fistful of salt
Seism graph
Schools are often held to account using data that can be misleading, says stats enthusiast and headteacher Michael Dix. Photograph: Olivier Morin/AFP/Getty Images

Warning: teachers - don't get seduced by the data


I recently accompanied a group of Year 3s on an overnight residential visit. As we waited in vain for the little dears to drop off to sleep, the conversation turned to matters educational. I was in the process of expounding on some issue or other when one of the other teachers said: "I've no doubt you'll be showing us a graph about it".

I was a little taken aback. Was this how the staff saw me: the producer of graphs, charts and assorted statistics, rather than the inspirational leader, the innovator, the motivator? A few retellings of this exchange back at school confirmed my worst fears. I had become seduced by data.

This irritated me considerably. I did statistics as part of my A level maths, and though the fine detail escapes me, I do remember that for data to have any significance there has to be a fair bit of it. What we spend hours deliberating over are such small samples that the margins of error can be huge: just take a look at the whisker diagrams in RAISEonline, the government's school data website.

I have always railed against the way schools are held to account using data that can be misleading, misrepresentative and manipulated. That's not to say that it can't be used to improve school performance, it's just that it needs to be taken with a huge fistful of salt.

The dangers of sample size were illustrated recently when a governor came to see me, having been on a training course about RAISEonline. He had been looking at our school's information, and while he was very happy with most of what he saw, he was a little alarmed that one of our ethnic groups had 25 times the national average number of persistent absentees. What was going on and how had we missed such a worrying development? What steps should we be taking?

When we looked more closely though, we discovered that the group comprised just three children and only one of them fell into the persistent absentee category: the same child has near perfect attendance this year. Crisis over, but the figures presented completely the wrong message.

Now, what do records of racist incidents, bullying incidents and unauthorised absence all have in common? Answer: they are all open to widely different interpretations.

Take racist incidents for instance. We do a lot of work in my school on tolerance and equality, and both staff and children are quick to report anything that may have racist overtones. Incidents are investigated thoroughly and recorded, and because of this robust and rigorous approach, we do usually have a few incidents to report each year.

Whether Ofsted would classify this as extremely rare, the criteria for an outstanding judgement, is open to debate. Some schools do report rare incidence, but are they more effective in promoting racial awareness and tolerance, or might some actually be rubbish at it, refusing to acknowledge racism as an issue and ignoring any incidents that occurred? Both will register zero returns.

Of course, the real trick is to know how to manipulate the data so that it shows you in the best light.

A secondary head friend of mine served for some time on the QCA advisory panel set up after the marking debacle of 2008. He was told by officials that since SATs became part of the overall teacher assessment at Key Stage One, the year on year rise in standards had levelled off in the vast majority of schools.

However, standards still continued to rise in infant schools. What a surprise. Infant schools are judged solely on these scores, so they will do all they can to achieve the highest possible results. With progress a key measure in all-through primary schools, it is in these schools' interests for KS One scores to be more cautiously given, so that greater progress can be demonstrated at KS Two.

So what is it that had changed me, Dr Jekyll-like, from an exasperated cynic into an over-enthusiastic zealot?

Could it be the steady improvement in our results over the last few years? Perhaps it is my love of mathematics and the fulfilment I get when I've managed to create a spreadsheet that will calculate the average rise in attendance of ethnic groups over the last three years, all colour coded of course.

Whatever it is, I am slightly reassured that from time to time a small voice inside interrupts my impassioned analyses and urges me to caution staff and governors not to get too hung up on the data because of its unreliable nature. And I reckon that 95% of them (based on a sample size of 48) understand where I'm coming from.

Michael Dix has been teaching for 34 years. For the last 15 years he has been a headteacher and is now the head of Glebelands, a 280 pupil primary school in the city of Leicester.


Could you be one of our bloggers?
If you have a success story, some good practice or an opinion you would like to share with our community of teachers and school leaders, please email emma.drury@guardian.co.uk.