Tuesday, April 30, 2013

Choice Quotes on the NYS ELA Exam

Many thanks to Lucy Calkins for having someone throw up a site to gather comments about the recent Common Core-aligned ELA exams in New York state. It is a bit of a mess, but kind of charming that in 2013 they chose to hack something together from scratch in PHP instead of just giving Google all the data directly.

Here are some quotes I plucked -- sorry they aren't attributed or individually linked. I spent way too much time reading the giant list of comments in the first place and couldn't get too fussy.

For example, two of the questions asked students, "Which of the following is the best summary of the article." For each of these questions, there were four lengthy summaries (a,b,c, or d). Students could easily narrow the four summaries down to two possible choices, but the differences between the two possible choices were so subtle that you're no longer measuring that student's ability to summarize! You're measuring if they can pick up on matters of inclusion and exclusion. It's more trickery than actually measuring the mastery of a particular skill, especially in this multiple-choice format.

The correct answers for those questions included a piece of information that the other possible summaries neglected to include, but at that point it was subtle enough not to even matter -- and it wouldn't in the "real" world in the context of how we use summary in our own writing. Summaries aren't meant to be nuanced!

State Ed. will argue that it comes down to "close reading," but in no way is this representative of close reading as its used in real world reading and writing! For 7th graders, it's a trap and not a true measurement of their skills. Questions like this set students up for failure, not success.

If I had a student read that same passage in class and then provide me with a summary of that passage's main points, they would easily master the skill. The multiple-choice format ruins this -- the format distorts the actual skill being tested.

I would argue, though, that the English Regents exam in 11th grade contains appropriate examples of multiple choice questions.

The current Common Core ELA State Test is so convoluted and sizeable that teachers will be forced to teach to the test – to teach isolated skills, drill those skills, and assess the acquisition of those skills all year long. This only deflates learning and teaching.

A multiple-choice test is not a good measure of the highest possible learning our students are capable of, and when the questions and responses are purposely written in a manner that’s convoluted, confusing, and imprecise, the format of the questions and responses is actually a form of trickery, a trap. Such questions trivialize the reading passages; such questions manufacture the illusion of right and wrong. They’re not measuring how well a student can closely read a full text, and they surely do not match up with how we think, read, and write in the adult world.

The State was able to undo an entire year of teaching kids to closely read in two ninety minute sessions. After the students used the reading strategy on the first reading selection, they realized there was insufficient time to continue close reading on the other passages and thus abandoned the stategy.
Thank you

On the Day 1 portion of the assessment, I looked over the passages and realized that some of the questions were not testing reading comprehension, but rather, were assessing our students' working memory. As I looked over a fifth grade test, I tried to find the answer to one of the questions. It required me to flip back and forth between pages to read several paragraphs to determine the theme of the passage. By the time I reread the passages, I completely forgot what the question was asking. With all the flipping back and forth and booklets falling off the desks, I can't imagine how my students will perform on this section.

However for 41/2 hours over the last 3 days days it broke my heart to watch some of our most gifted 5th graders trying so hard. They read and read and read....they wrote and wrote and wrote....the entire 90 minutes. I watched them shake out their hands and wiggle their little fingers to reduce their hands from cramping. They were checking and double checking their answers, underlining and circling text, everything they have been guided to do in their lessons leading up to this week. No break for 90 minutes, except to hear me give them them the 10 minute warning. On Tuesday I had 4 of my top kids not get past 30 out of the 42 questions. Wednesday, 7 of my Top kids never completed the final essay, one asked me which was worth more because he wanted to tackle that one to get more points. However, today was the hardest for me to swallow... in the final minutes I watched a child that attends young scholars be reduced to tears from the fact that he could not finish for the third time.

In my classroom, I also witnessed the opposite situation. I had another young scholar student who is bright beyond his years complete Test 1 which includes 4 reading passages and a poem as well as 42 multiple choice questions. He had "completed" his test in 35 minutes. The rest of the time he put his head down and slept. On day 2, he completed both books in less than 40 minutes. As I collected the tests he verbally expressed to his 8 peers that had not completed the test, "don't worry because these tests were only going to be used to grade the teachers." I did have a long talk with him, as well as calling his guardian to help guide them with the facts so that they understood that this test WILL effect him in some way.

At the end of the day today, the parent of a high anxiety child came into the building with the sole purpose of giving me a hug just because she knew what her child had been reduced to and we had tag teamed to keep her going, me in school - mom at home!!....I had no way of holding back my tears at this point.

I was shocked by how much the test measured inductive and abductive reasoning with overwhelming cognitive complexity, and how little it measured the standards appropriate to grade 3. The answers did not follow with certainty from the questions, and about half of the questions themselves appeared intentionally vague, syntactically and semantically complex, and/or challenging with regard to eye-hand-coordination. Children had to cognitively fill in the questions with what did not exist, or sometimes even ask themselves what the test makers REALLY wanted to ask because it was not readily apparent. Even adults often arrived at an "if, then" but "if, then" range of responses that then required internal argument to choose. Surveying the premises required to answer the questions made me and my colleagues exhausted by about a third of the way through Day 1, and we are not eight or nine. Early on children I observed became confused and exhausted. They persevered with their best intentions and innocence, lacking the experience to say to themselves, "This is $#@*" I thought the second piece was about a level R/S becasue of embedded metaphor, central to both questions and meaning.

Residual fatigue factored into children's performances on subsequent days. I never witnessed so few children writing "their best game." With all of Pearson's test creation experience and generous funding with our tax dollars, they made a test that is completely developmentally, cognitively, and syntactically inappropriate to the population.

If I stay in teaching, I will be writing my own improvement plan any second to propose to district leaders. It will say things like:

1. Get children to practice flipping back and forth between stapled pages while holding a question in their mind for at least 10-15 lessons if you want them to pass the ELA. This is a demanding and complex task that requires direct instruction for success.

2. Have children read questions intended for linguistic and intellectual screenings over and over, about two years above their current grade placement for 20ish lessons. They may or may not be able to learn how to understand these questions, but at least they will be familiar with increasingly complex questions.

3. Teach children not to become agitated, upset or apathetic after 10-20ish questions that they cannot easily respond to. Breathing and meditation for at least 20-30 lessons.

4. Stop accepting into the reading program: children with below "above average" IQs, ELLS, the emotionally challenged, and ADD children, because really, they don't have a chance.

The only recommendation for my teacher/ teacher-trainer improvement plan that has anything to do with the CCSS is:

5. Get more teachers to teach children to look at word choice, synthesized with tone, mood, and author's purpose in regard to setting.

I sat at a table with grads from Cornell, Columbia and a Brown U. attendee who argued tenaciously about answers.

Is Cuomo's agenda to make most teachers fail? So he can look better when the test gets easier next year? It certainly was not Pearson's purpose to test the CCSS. That I know for sure.

Although kids were asked to read nonfiction passages, they were not expected to read them in order to learn about the topic. Instead, the goal was to analyze the structural and language decisions, to label the craft moves that authors made. That would have been fine if it had been 10% of the test, of 20%....but it was closer to 70%. Tests end up being the tail that wags the dog, and they shape curriculum.

1 comment:

Leroy's Mom said...

Thanks for providing me with material to get up off my lazy ass and write something...