Testing 4

In a posting last September I briefly discussed the notion of letting a test be a learning experience rather than simply an assessment. The three most important parts of doing so were 1) setting essay questions in advance, 2) letting students take time to guide themselves through preparation for an answer, with the teacher ready to pitch in at need, and 3) actually writing the essay after a momentary surge of excitement and interest caused by a lottery to choose which question to answer.

I want to return to that subject because of an article that recently appeared in The New York Times. It reviewed research showing that students remember what they “learn” better by being tested on it than by working up “concept maps” or by doing little review sessions. It is important to note the article’s summary of the experimental protocol: “Without the passage in front of them, they wrote what they remembered in a free-form essay for 10 minutes [emphasis added].”

A moment’s thought will tell us why an essay, even a “free form” one, is intellectually both more demanding and more rewarding than a concept map. A simple line drawn between two words or phrases suggests a relationship per se but tells us nothing about it. By contrast, even a “free form” essay is not so free as to dispense with the parts of speech, which means that it will link concept-words with prepositional phrases, clauses, and conjunctions that express in some fashion exactly what kind of relationship those words have. Drawing lines is by comparison a barren evasion, a mini-holiday from responsibility. Even less thought will tell us why an essay is superior to little review sessions, which so often amount to nothing more than students rattling off details to each other like the young George Orwell conning his history facts with black-negress mnemonics. How many minutes after the examination for the Harrow History Prize did he remember the names of the battles in the War of the Roses?

The article says that “why retrieval testing works is still unknown.” I have a couple of possible explanations. The first depends on my understanding of William James. According to him, “what interests us is real,” but the world of reality is actually “Many Worlds.” After taking an interest, however cursory, in something, we place it in one of reality’s Many Worlds using our judgment. He says further that we place these worlds in a hierarchy of importance that depends on our temperament and mental inclinations. In effect, some of the Many Worlds interest us more than others and are therefore more real.

Most people do not rank the world of abstract relationships high among the Many Worlds, so that when Bertrand Russell reports in his autobiography that learning geometry was like being in love for the first time, most people do not share his delight in that reality. They are not nearly so interested in abstract relationships as Russell, and we don’t fault them for it. Most people think of abstract relationships in connection with concrete or sensory problems when they think of them at all. It is strange in this light for educational psychologists or cognitive scientists see the “gold standard” for examinations in concept mapping, a technique that puts a premium on the establishment of abstract relationships, sometimes expressed in a line without a name and therefore without a “stinging term,” as James calls those interesting little bits of reality that compel our attention.

To understand why writing an essay test works better than a concept map, we should revise what we think of the “cognitive and affective domains” of learning that we heard about in teachers’ college. Cognition and affect do not have separate domains: they are the condominium of intellect. When we are interested in something, we cognize it better, more fully, and with greater relish than when we are not. The networks of interest that we establish run from their high points in the worlds we cherish down through other worlds to those about which we are indifferent.

Good teachers complement students’ predilections by establishing extrinsic patterns of interest to connect the Many Worlds, just in case not everyone wants, like young Russell, to fall on geometry with a tiger’s appetite or, like the young Samuel Johnson, to read thousands of books by his eighteenth year. One interest universally established is in the results of a test. Whatever else we have to teach high-school students, we usually don’t have to get them interested in a test grade. (The student too flat and feckless to be interested by anyone in anything is another matter.)

A teacher who sets essay questions has two things going for his students’ learning: one is the interest of the question itself, and the other is the interest provoked by the situation of the test. When the student works up notes on an essay answer and then writes it, he is propelled by test conditions and his interest in the question to establish the concept-work needed to give a shape to the details and in complementary fashion to use those details as the factual anchor of his ideas. He expresses relationships in language not lines, thereby getting them more exactly. He has chosen the language for an exercise in which he has an interest. All these things work together to make the material more memorable than it would be as a jumble of detail or a wind of unsecured ideas. Used correctly, testing doesn’t just examine thought and memory: it secures them.

But the testing-and-accountability people should not think this insight vindicates their approach. Their testing is not an invitation to the exercise of directed and participatory intellect characteristic of an essay exam with a generous run-up. The competency tests favored by them do not invite anything except pointing (at The Right Answer). They are entirely summative, not at all formative; and they are summative in a bad way.

The constructivist Howard Gardner rightly sees the results reported in this article as a challenge to constructivism, but surely a large mind is capable of recognizing the challenge as an opportunity to enlarge the possibilities of the classroom beyond the constraints of a single approach. Essay questions and other tasks set by teachers combine direction by teachers and construction by students in a synthesis worth pursuing. Surely there are more such possibilities?


Phillips Grand Central

Those of us who believe that memory is more than a test-taking skill also have a rooted conviction that it works with other mental powers, sometimes for simple delight but also as a guide. One of the oldest ways of grouping these powers was to call memory and its allies the Five Wits. Of those five, I want to consider imagination and common sense as helping us with our memories to take good educational decisions and actions and to avoid bad ones.

Memory not serving for much more than threescore and ten years, we have developed our imagination and common sense (could we in this case call it worldliness?) in the discipline of history. That, combined with the records of annalists and chroniclers, either as written or as shaped by need and culture for transmission, is what Richard Hofstadter called “the collective experience of the human race.”

When the collective experience of the human race tells us something, we ought to listen. The same with history. It is therefore exasperating to read that in Brooklyn a “new experiment” is in progress that appears to be ignoring these lessons. This “new experiment” is none other than an Open Classroom.

Against history, common sense, imagination, memory, and the collective experience of the human race are ranged a handful of Harvard graduate students and Joel Klein. They seem bent on confirming the critic George Steiner in his dictum that “American education is organized amnesia.” It must be, to have forgotten the open classrooms of forty years ago.

But when memory won’t serve, we could use imagination and common sense to supply the mental shortfall. The ostensible purpose of this open classroom is to duplicate the educational successes of the Phillips Exeter Academy, “where students in small classes work collaboratively and hold discussions around tables.” So they do, and well, but we must think and imagine carefully before bringing Exeter to Brooklyn.

First, there are physical and organizational differences between a classroom at Exeter and Grand Central Terminal. It is possible to conduct group activities in Grand Central, as a number of successful restaurants show, but I would not want to generalize from this success that uninstructed and incontinent six-year-olds in a cavernous constructivist environment will get their tasks as successfully as commuters headed to their tracks or as menu-driven diners at the oyster bar or the steak house.

Second, the students at Exeter were eminently successful pupils at other schools and have internalized many, many lessons in conduct, culture, and academics that a hangarful of first graders would not yet have managed to assimilate.

Third, classes at Exeter are generally limited to twelve students (not just any twelve, but twelve focused, highly motivated, extensively trained, and disciplined young—but not too young—people). Anyone who has taught classes of twenty-five or thirty and classes of twelve or fewer students, as I have, knows that there is a world of difference between the two. It is suspect, if not reckless, to assume that successes with a room of twelve in Exeter can be duplicated in a room of sixty in Grand Central.

Common sense and imagination tell us these things, but the open classroom also has the lessons of history against it. Open classrooms were tried, and they failed.

It is pointless to argue that they might have succeeded if they had been conducted by teachers who leap tall buildings, walk on water, and assimilate the findings of second-rate research: most schools don’t have many such teachers. That argument is another version of the old wheeze that the idea wasn’t misbegotten, but that instead the teacher was to blame.

The impetus to blame also seems to have governed another decision at this “academy.” I mean the assignment of students to single teachers in successive years. Is it done, as it ought to be, because there is an educational benefit from knowing whom you teach? (And there is.) No, it is done to increase teachers’ “accountability.” All that this assignment will end up showing is that bad thinking sometimes accidentally has good consequences. On a lark, here is bit of accountability-thinking: whenever Joel Klein or anyone else cooks up and imposes a scheme to improve learning and it doesn’t work, let them have their own salaries cut by 20%. But even better than waving paring knives around the schoolhouse would be genuine collaboration based on respect, the Five Wits, and the collective experience of the human race.

Diane Ravich reports that one special school in New York has had remarkable success in teaching disadvantaged children. Unlike the Grand Central Academy, this school has boarding students and a highly structured program in small classes. She notes what this school does not often report: that the assistance it receives from private foundations allows it to spend in the neighborhood of $35,000 per student per year.  This is about what the Phillips Exeter Academy charges.

Common sense, imagination, and experience tell us that the commitment of ample resources to teaching in a program using methods that history shows have worked will be more productive than teaching on a shoestring in a hangar, which history shows has not worked. If I were a betting man, I’d rather put my money on Exeter and the Harkness Table than on Grand Central with beanbags.


Hardest Hue to Hold

Look, up in the office: it’s a bandwagon! It’s a juggernaut! It’s education by the numbers with accountability! This movement, combining the best managerial principles of Lady Bracknell[1] and Marshal Stalin[2], says that any statistics are better than none, that statistics are normative not descriptive, and that when a principal or teacher is shown by dubious statistics to be doing, or presiding over, a bad job, the best decision is excision.

But there are alternative and truly effective ways to produce good results, and so here are stories about people of my acquaintance—principals, other administrators, teachers, and, yes, even a consultant, who worked outside the Bracknellian-Stalinist paradigm to bring good education to their schools. Historians, anthropologists, and parabolists all know the educative and explanatory power of good stories judiciously told and used, so why not educators?

One day back in the 1990’s my principal and I went to the Orange Farm “informal settlement” (shantytown) southwest of Johannesburg. We were in a large truck carrying unneeded school supplies to the Leshata Secondary School, which had made news because its graduates had scored between 95 and 100% passes in the “Matrics,” South Africa’s comprehensive school-leaving exams. The results were unprecedented, but after we arrived, I came very quickly to understand what was happening.

The Principal, Mr. Moeketsi Molelekua, welcomed us at the gate. When we came in, we discovered that the students, all in uniform, were coming out to the school’s assembly-ground (no auditorium: too expensive) to greet us too. The students, most of whom lived in shacks without heat or electricity, stood in files to gave us a traditional black South African welcome with thanks: they sang in Sepedi, the predominant mother-tongue. The singing was joyous, full-throated, and beautiful. My principal and I briefly thanked them for their welcome, and they listened attentively and silently even though (we knew) most of them could not understand American-accented English.

After this welcome, than which a more moving one could not be imagined, Mr. Molelekua immediately gave a speech that turned out to be more moving. Within about ten seconds I understood why this school was getting 95% pass rates. Mr. Molelekua had an extraordinary charisma, forensic power, and eloquence, which he used in the service of his belief that education would transform South Africa and improve the lives of its people, in particular the students standing before him.

During out talks after the assembly (his office had his own modest desk, two plastic chairs for visitors, his assistant’s table, a small copy machine used sparingly, and a computer without an internet connection), I realized further that Mr. Molelekua’s belief was strong and genuine. He himself achieved a university education under the old regime in spite of economic and legal impediments while holding down a full-time job and supporting his family, and he pictured his students as in some very important ways exactly like himself.

He and the teachers kept the school open after hours into the evening on school nights so that the children, who often had no light or heat to study by and were crowded with their families into single-room shacks, would have a place to do their work. Believing that parents’ help and advice was essential to the operation of a good school, he would regularly see up to forty parents a day, seeking them out when they didn’t come to him. He invited parents to help him choose new teachers, and he accepted their advice. He visited the neighborhoods served by his school, where he was well known to parents and other members of the community.

In short, he was an educational leader, not a businessman. While I doubt that any administrative credential could confer on anyone the qualities that Mr. Molelekua brought to his calling, I feel saddened and upset that in the U. S., states and school districts seem to be moving away from the belief that educational leadership, however recognized, is what is needed in a principal or a superintendent, and instead that a programmer or a publisher with a plan and a knife will know better how to run a school or a district.

(The sad turn on “nothing gold can stay,” if my information is correct, is that Mr. Molelekua is no longer at Leshata and that the school’s results have sunk to ordinariness. I hope it is not true. Part of the reason that “bliss was it in that dawn to be alive” in 1990’s South Africa was the unpretentious but white-hot conviction of (extra)ordinary people like Mr. Molelekua that their decisions and actions would make a difference in their and their children’s lives, and that these beliefs, not statistics, were their guidance. So dawn goes down to day.)

But good education need not be the result of decisions by administrators. I had the happy good fortune to work at a high school that over a period of eight or ten years developed a remarkable productive collegiality among teachers. It was as if a roomful of Messrs, Mmes, and Mss Molelekua found ways to transpose their convictions into good teaching and learning without a single administrative ukase. Our principal generously kept to a laissez-faire style of running the high school because that was exactly right given the conditions. Some of the credit was due to the ingenious consultant Mr. Martin Skelton, for his approach respected teachers and gave them the time they needed to embrace his suggestions, which included a plan for the evaluation of teachers not by administrators but by their colleagues. We also devised a program of “writing across the curriculum.” We had a wonderfully varied offering of “activities” for our students in addition to the core of academics. My own contributions included help in the revamping of the English curriculum, work on the adoption of reading and writing assessments that we devised and graded ourselves (we used them to assess our programs, not our students and certainly not our teachers), supervising the adoption of “writing across the curriculum,” and running an outreach program to a nearby orphanage. I also set up a “TOK trip” on which students taking Theory of Knowledge camped by a riverside and mixed fun with work on their culminating “TOK Presentations,” which were given in a conference setting outside the classroom. Ms. DM, the I.B. Coordinator,  taught physics, rode horses, and climbed Mt. Kilimanjaro with students when she wasn’t running the I.B program. Mr. JW established an international Model United Nations conference. Mr. BK, the Activities Director and P.E. teacher, helped run the MUN conference, set up a program in which NBA players met with local disadvantaged students in a kind of basketball clinic, and organized invitational tournaments in I forget how many sports. Mr. OF, an Olympic medalist with a Ph.D., made student government real for many years, except one, during which MW, a clever and energetic stinker, was the student body president.

It is important to see that the good work described here has nothing to do with strategic planning or business models. It has nothing to do with statistics. It has nothing to do with elimination of undesirable elements. Instead, it has to do with educational leadership, whether by inspired principals or by devoted teachers. It has to do with verve in the service of deeply held beliefs. It has to do with respect and nurture that regard the materials they work on as live and fragile and not as lumber to be hammered, nailed, or burned.

[1] “Statistics … are laid down for our guidance.”

[2] “No man, no problem.”


Chinese Students

Now that students in three Chinese cities have gone to the head of the international class in the PISA tests, we are starting to see coverage of Chinese education, much of it tinged with a trace of sour grapes. 1) “Schools in Shanghai are better than the average school in China.” 2) “Chinese students all learn by rote.” 3) “Chinese students are concerned only with the test.”

1) This must be meant to suggest that China’s scores wouldn’t be so high if they included test-takers from smaller cities and rural areas and that picking and choosing cities gives an unfair picture of Chinese education. That’s true, but it’s beside the point. Comparing China and Europe: we have focused on Finland’s schools as establishing a world-class standard of education without worrying that other schools in Europe are not as good as Finnish ones. Finland’s five million people run better schools than the rest of Europe, so they may have lessons for us. What about Shanghai’s twenty million? Or is the point of PISA testing a competition?

2) This may be true, except when it isn’t, but it bears more investigating. In a pedagogy that recognizes the differences among knowledge, skill, and understanding, rote learning has its place and should not be sneered at per se. The question should be Is rote learning used appropriately and with discretion? A refinement of that question: Is it complemented by coaching (skill) and Socratic probing (understanding)?

3) It is probably true, though not universally. The poet Du Fu (712 – 770) took the Civil Service Examination twice, failing both times. The failures didn’t keep him from writing over fifteen hundred poems, and at one point he even worked as an education bureaucrat, writing test questions for exams, a job he hated. One explanation of his first failure is that he refused to modify his dense style to suit the tastes of his examiners. The generally accepted explanation of the second failure was a case of result-rigging by a suspicious office-holder. But even if Du Fu is an exception that proves a rule, how different is that rule from the one governing test-preparation in the Land of No Child Left Behind?

Let me add some observations that are the result of having taught at a high school in China:

•       Chinese students are strongly inclined by upbringing to respect their teachers. September 10 is the national Teacher Day, and it is widely observed. It is impossible to explain briefly how this one attitude washes away mountains of crap from schools.

•       They are highly mindful of their studies. Electronic entertainment and distraction are making some inroads, but generally, when Chinese students study and pay attention, they do so successfully. This success is the result of good habits’ being instilled early and proved often.

•       Their parents are generally the allies of their teachers, supporting teachers and encouraging their children to do well. At parent conferences the student often attends, sometimes translating English into Chinese—accurately, even when the teacher has a reproof to deliver.

•       They tend to see schooling as a mission to accomplish rather than an annoyance to be endured.

•       They are sixteen in many of the same ways that high-school students are sixteen around the world.