Sunday, 14 October 2012

Why we need Evidence: part.1 'it works!'

Teaching isn't the most rigorous of professions.  It's not glamorous and usually not very well paid. Most of the teachers I meet do it because they love the job, and they love the students.  It's often said that teaching is "an art not a science".  There might be some truth in this. But is evidence unimportant?  I'm going to try to argue 'no' in a series of posts. 
It's difficult to prove much of anything in TEFL and there is very little for which  there is solid evidence. However, new techniques and approaches appear all the time and are taken up with vigour by teachers who become convinced that this time they have hit upon the holy grail of teaching -the method to rule them all!  They are sure that this time....this time...they have discovered the method that will turn their barely communicative disinterested students into fluent autonomous learners. Said teacher is convinced of the efficacy of the approach due to the stunning results it produces and the expressions of sheer joy on student faces. This position could be called the "It works- just look at their faces!" position. A few examples are the following:

"Of course we all know Genki English works great because we see it every time on the kids’ faces" (2009 online) Richard Graham, founder of GenkiEnglish, presenting the 'evidence' that his method "really, really works"
"both kids and teachers told us it really works" Video extolling the virtues of Mindfulness training in classrooms.

Teachers using BrainGym continue to this day, despite all the evidence against it, continue to
insist that it works.
"In the final analysis, like any other methodology, [neuro-linguistic programming]NLP will work or not for an individual teacher because it is right for them and not because it is scientifically proven or not." (Harris 2002:37)
I cannot really say that these doubts have completely disappeared but I can say that ,little by little, I BELIEVE that the magic of NLP can actually come true. I am really conscious that the changes I am experiencing with myself and with my students in class...How do I know is it working? Because I can see it in my students´faces, gestures and attitudes (Esteve online)
Now some teacher might take me to task here saying "well how can we prove whether a method works or not in any scientific sense?"  This is a fair point and I agree but I have two caveats to add to it.  Firstly, if a method can't be proved to work, then we should resist saying that "it works".  Certainly we should not suggest that students' reactions or the way we feel about it constitute any kind of reliable evidence.  Secondly, though it may be difficult to prove that something works, it's relatively easy to prove that something doesn't work, -or can't work.  for example, NLP claims that you can tell a persons "learner style" by watching their eyes move and listening to the pitch of their voice.  BrainGym claims that children can children can massage their bodies to increase the oxygen supply to their brains.  Both of these claims are demonstrably false
Carl Sagan's baloney detection kit, is a good place for teachers to start.  In this case, the following principle might be useful:

wherever possible there must be independent confirmation of the facts

This is because people believe what they want to believe.  A teacher who likes a a particular activity/method/approach will find it easy to convince themselves that their students like it, or benefit from it.  Confirmation bias (i.e. recording the hits and forgetting the misses) will do the rest to convince a teacher that their method "really works".

But what does "work" mean here anyway?  If you want to test something then it's a good idea to have a clear idea of what it is you want to test.  Does "work" mean "make the students happy" or "allow me to skive off the lesson" or "make me, the teacher, feel good about myself" or "increases the chances the group of students will become more proficient"?  If you don't know what "works" means then it's meaningless to say that something works. 

no one is impervious to this kind of thinking, which is why we do need evidence that our practices work, or at least, the ability to weed-out those which really do not.  Fifty years ago teachers were making their students listen and repeat and declaring that "it really works!" and 20 years ago communicative language teaching came and that "really worked" too and now Dogme "really works!"  If all these methods work, why do we keep changing them?

part 2 is here
for more about GenkiEnglish read this.

NB:  If you want to read a blog which basically says everything I do, except funnier and before me, then check out this one.


Harris, T. 2002. ‘NLP: If it Works, use it … or is there Censorship Around?’ in HLT magazine retrieved September 23 2012

Graham. R, (2009) Academic Research: Genki English really, really works. In Genki English. Retrieved May 7 2012, from


  1. Another great article. I always approach these claims with a grain of salt, but I've made some bad picks in my day too.

  2. Thanks for the comment! Haven't we all?

  3. Excellent post. I firmly believe that we, as teachers, must always be examining what we are doing. I also believe that it is misguided to ever believe there will ever be "one method" that "works".

    Great job raising the question and pushing us to never be complacent!

    John Pfordresher

  4. Thanks John!

    I agree that we may never find a "perfect method" Especially as teaching contexts can vary so much. But we can certainly prune away those which can be shown to do do little or no good.

  5. I'm all for evidence based teaching. But Instead of beating down methodologies that apparently lack evidence, what are we doing to find or evidence methodologies that do work?

    1. excellent questions, - do you have any ideas? could we test them?