June 4, 2008

The problem with education research

Eduwonk asks his readers what would they do with $5 billion to improve American education.

Many responded that they'd funnel the money into various education research projects -- some sensible, some not so much.

Show of hands. Who here thinks that education research will improve education?

If you answered yes, do you know about Reading First?

If you know about reading First, how could you have answered yes?

Our problem isn't necessarily the lack of quality research. Our problem is that schools often aren't following the research (for various reasons).

For example, high-quality reading research has existed for quite some time now and points to very specific teaching practices. Educators, however, have not voluntarily adopted these research-based practices. Why should they? They like what they are doing. There is no incentive in the system to improve or get better results. Results don't matter. This remains true for most schools under NCLB.

Reading First was a top-down legislative attempt to bribe educators into adopting these research-based practices. An attempt that did not survive the legislative process. In the end, the legislative language was so watered down (intentionally mind you) the vast majority of funding went to reading programs with no research base. Some legislators didn't want funding to be distributed broadly instead of going to the few reading programs with their own research base. Other legislators wanted funding to go to reading programs of their political supporters regardless of the program's research base. (Let's not even get into the completely ineffective research that was commissioned with Reading First funds which will likely yield no usable results.) In the end it was politically impossible to enact such a law, even though the law had broad bipartisan support.

Right now the best indicator that education research will continue to have no effect on education policy is what the publishers of all those bogus reading programs that received funding under Reading First are doing. Or rather, what they re not doing -- they are not conducting research on their own products, despite reaping a huge windfall from Reading First. They're doing this because research doesn't matter. It didn't matter before Reading First. It didn't matter under Reading First. And there are no signs it will matter after Reading First. In short, why should publishers waster profits on research that ultimately won't affect their sales.

Do you still think education research matters? Tell me why.


Anonymous said...

A contrarian example...

Any research would be accomplished by current members of the monopoly from within the body of the host species. The likely outcomes of said research would tend to be supportive of the continued existance of the host.

My vote? More research = less change!

Rev. Mike said...

Hey, wait a minute ... I recognize this ... this is a quiz! Who here has been reading your blog long enough to remember the posts you've done about the utter ecstasy edu-researchers derive from mere tenths of a standard deviation improvement when any serious scientist/researcher in any other field would insist on 2 sigma? Well, I guess I pass because I have passed those posts around to more people than I can shake a stick at in order to prove my point that the educators don't stand a chance when their colleges persist in teaching pedagogies no self-respecting researcher in another field would consider valid.

My vote--edu-research won't improve diddly.

KDeRosa said...

paul b, I don't think there is a need for the monopoly to do any research to support the status quo. In any event, what the monopoly tends to do pseudo-research designed to achieve the result it wants. See the Reading Research Quarterly for ample examples.

rev mike, you pass.

Kim said...

Rev Mike nailed it. The educationalists are tied to their Dewey and Rousseau philosophy in the face of reason and evidence.

Anonymous said...

You cite Reading First as the ultimate example of how education research is a failure, and then say that Reading First failed because of the legislative process.

Education research is a tool. It can be good or bad or completely unnecessary, in the same way that a school might benefit from having computers, or the computers might be implemented poorly and become a waste of time, and technically the school can run without them.

If you think that education research is too difficult to do correctly, then say that. If you think that current education research is overly focused on aggregate data and doesn't study individual students enough, then say that. If you think that education research tends not to provide useful enough instructions for implementation, or if you think legislature / school boards / principals / teachers tend to use the research incorrectly, then say that.

But saying that a tool, on the whole, is useless just makes it sound like you don't understand how to use the tool. You're being anti-academic when you mean to be anti-lazy researchers.

V01-C39 said...

A no-nonsense, no skipping, no guessing approach to reading readiness -- WordsAhead.org -- is waiting to be tested!

I dream of a Principal who will allow even one teacher to choose-and-use a speech sounds approach with Word Lists for Literature Mastery (sound-sorted author vocabulary) prior to shared inquiry (comprehension discussion).

Get your most qualified edu-researcher to compare and contrast. Truth is not the enemy.

Anonymous said...

In response to anonymous: I thought the original post was quite clear, and I agree with the main point--the problem lies in the system. There is no motivation to change practice based on the recommendations generated by research, and all the usual reasons not to do so (foremost amongst them, it's a pain in the neck). But I would be harder on the research community than Ken was. There is some excellent research, but very little on the really hard problems. For example, the overall conclusion of the 2005 AERA summary of research on the effect of teacher education was "there should be more data."

Brett Pawlowski said...

I think education research can make a difference - the problem is that the good research hasn't been going to the right people.

Take EduWonk's $5 billion dollars and do saturation outreach (TV, radio, direct mail, online) on Project Follow Through to ensure that every citizen in the country knows and understands the project and its results. There should be enough money to also let people know which curriculum their local school uses, and where it falls (even if by another name) on the scale. Then see what happens.

I won't guarantee results, but I think it would have a lot more impact than just about anything else.

Heather said...

I don't think that research is the problem. Without it, we wouldn't have recognized the demographics on either side of the achievement gap, after all. I agree, however, that many teachers don't want to implement what research proves.

Granted, there's always research that disproves other research, and granted, there's always the data of the month that is brought in by every new administrator as a means to put their mark on a school, but I happen to agree that there is a contingent of teachers out there who don't want to do the most obvious of researched and tested strategies.

Collaboration, for instance, is harder then isolated teaching. As a result, you get two groups in every school: those who will and those who won't. Differentiation, on one level or another, is harder then whole-class, teacher-centered learning. As a result, you get those who will and those who won't.

It isn't the research that's the problem. It's the teachers who won't step up and evolve based on it. Sure, not all the money can go to research. And, sure, if I were only given 5 million dollars, I sure as heck wouldn't spend it on research, but education has many needs and research can meet some of them.

Now, if you want to talk about using that 5 million to AUDIT education (at every level, mind you: federal, state, district, and site), well, now, there's money well spent.

Robert Sperry said...

This is unduly pessimistic, I will not offer you my lack of sleep induced opinion

Should we stop research into evolution because the fundamentalist, and animist wont be converted? While I agree with the premise the conclusion does not follow, because there are some in education that do look to the research and research based products. Even if its just a small slice of home schoolers, and one percent of everyone else and some people in Singapore.

I mean do you want to stop all research into food just because the last farm bill was so idiotic?

Besides who buys stuff based on the research? The research to create, validate, test etc most products is not even public. This is true for most engineered products. Research and development is done but no one buys it after reading the research. But thats becase the products have a long track record of working, and when they don't they will replace your broken part.

I am sorry but your algebra class didn't work for my son, can you give him 3 hours a day back for 6 months thanks. So there is even a bigger incentive to get education right the first time.

What is lacking in education is engineering. Sure its fantastic when scientist discover something, its cute how they get all excited and start projecting what it could turn into. But lets be serious, its the engineers that do the turning.

How many engineers work in designing, building, and testing educational materials in a manner similar to any other engineered product. 3? 10? 100? How big is the annual DI research and development budget? Outside of this how much have things improved since Feynman blew up in his basement after reading grade school math books?

So how do we spend 5 billion?

Discovery: look for great teachers, process, what is working, go far and wide
Theory: Try to develop a theoretical framework or several to explain stuff
Experiments: Try to actually disprove said theories etc
Measurement Technique Development: You can not improve what you can't measure

Build Something: Take what you are most certain of from above, and create products, be they books, curriculum, computer games what ever.
Process: Follow a engineering process to build, test, and fix the product. If they don't actually work cancel the project, don't just ship it this isn't software you know!

Social Science
Follow on testing: Now you can do product comparison testing with those nice longitudinal studies

Manipulation: How do we get, students, parents, voters, teachers, politicians, the people that bribe-donate money to politicians to use good materials and process?

For only 1 billion I'll fill in the rest of the details.

KDeRosa said...

I must not have been clear.

I didn't mean to suggest that research lacked value or shouldn't be conducted.

Rather, what I'm saying is that there is no incentive in the current system and no political means to force educators to adopt the research.

If you think otherwise, you need to explain why almots all of the extant research remains unadopted on a mass scale.

TurbineGuy said...

I disagree. I think a new Project Follow Through study could have a significant effect for one reason. Media.

In the 60's and 70's it was much easier for the education establishment to hide/dilute the result because access to nation wide news was limited to the local newspaper and a few new programs on the television.

I have asked every single teacher any of my kids have had about Project Follow Through, and not a single one of them had even heard of it. This goes to show how effectively the program was buried.

A large enough, and publicized enough, study today wouldn't be able to be ignored.

Robert Sperry said...


We agree here.

I just wanted to carve out that there are a small minority of people and institutions that do apply the research.

Education is where medicine was when James Lind proved by experiment that citrus fruits could cure scurvy.. it only took 30 years for this to be put to practice and not for 50 years before it became fully embraced.

But my kid is four and a half, so I am in a bit more of a hurry.

Anonymous said...

Yes, I think some research does matter. For example:

Here in this small Florida GOP town -- where the one daily newspaper does not do any investigative reporting whatsoever -- we have a new superintendent. He is a retired Army colonel -- and I think he has fake educational credentials.

Should one research this matter? I think so.

Consequently, here is the result: a short, online, 10-question, multiple choice quiz I made about his actual background, based on publicly available information. You can take my quiz, and then decide for yourself if this guy is a fake:

Dennis Thompson Background Quiz