Mr. Stickfigure is back, and more confused than ever. What the hell is it that I teach?
On the students’ report cards it says ENGLISH. On teacher programs it says ELA, short for English Language Arts. And according to a million pages of pedagogical confetti, I am a teacher of literacy. This last title will be my straw man for today. And believe me, this scarecrow is a fire hazard.
The New York State English Language Arts Exam is a devilish assessment. When I started teaching, a grizzled English vet said of the ELA, “And now they’ve made this fucking test that can’t be cheated!” Since he was the type of bilious, weary pessimist I flattered myself to despise at the time, it took me three years to see how right he was. That was when I spent my first week scoring the written portion of the ELA in the district office.
Scoring the ELA is a secure and ritualized affair. Official scorers are never within arms reach of the exam papers that come from their own school. Instead, they toil in small clusters, comparing the work of anonymous students to rubrics and anchor papers. After enough hours of scoring, you begin to feel the data moving through the computer part of your brain—the supercomputer part, the part that is the envy of the merely electronic device that scores the multiple-choice questions.
Human grey matter is the only computer than can perform the calculations necessary to score written exams. But make no mistake, it is calculation we perform, and our output is as pure as raw, binary data. All of this talk about the subjectivity of human scoring is based on a misunderstanding of scale. We marvel at the objective wonder of computer-aided assessment. What we are forgetting is that even computers can’t handle raw, binary data.
It takes an incredible amount of sophisticated redundancy to get a computer to reliably distinguish between A, B, C and D. The first million operations may be flawless, but before long, there’s a glitch. An electromagnetic surge, intermolecular friction, sunspots. A 1 becomes a 0 and a B becomes an A. The reality of data loss is actually what proves the wonder of our technology, which is over-engineered to withstand the loss. It is hard enough for a machine to master its ABCDs, which is precisely why we can’t trust an essay to an insentient computer.
Yes, when you score the written portion of the ELA, your emotions do occasionally swell and cloud judgment. You champion an iconoclast, give the benefit of the doubt to a kindred spirit, strike down a boastful persona. Occasionally it happens, and skews the results. Given the aggregate complexity of the calculations being performed, these subjective lapses amount to an acceptable margin of error. Meanwhile, the human scorer’s brain is leased for the processing of raw data.
After enough scoring, you begin to see the minds of the children behind the papers. More, you begin to see their classrooms, their teachers, their bulletin boards. You see their halls and their auditoriums. Finally, you see all of the way out onto the street and right back into their homes. Yes, these visions are colored by your personality and your prejudices. But when you score the ELA, you’re not judging homes and streets and halls. You’re not even judging classrooms or teachers or individuals. Your task is a simple as it can be made: Decide, on a scale of 1 to 4, to what extent the words on the page imply the students’ ability to express their reading comprehension in written form. And don’t let the generality of this description mystify you. If you’ve made it this far, you would score a 4 on the New York State ELA. It’s as simple as that.
For official scorers of the ELA, panoptic and panoramic visions of the educational landscape are a byproduct of our relentless computation. These images may exceed the margin of error, but they are enough to get a sense of the big picture. What they’ve done, like the old goat insisted, is make a test that tests how well you’re educated. Mr. Stickfigure is here to tell you that his astral presence has witnessed a hundred teachers try every trick in the book to raise their 1s and 2s to 3s and 4s. No dissection and reconstruction of the test and its peculiar format, no battery of last-minute strategies, no school, district, city or state-wide assault on the ELA itself will budge more than a few 1s and 2s. The only statistically reliable assurance of 3s and 4s is a good education.
Which is what makes literacy instruction so confusing to me. Suffice it to say that for teachers of 1s and 2s, the shadow of 3 and 4 looms large across our path. In this metaphor, it is the ELA itself that causes the eclipse. Knowing that light is on the other side, we stare into darkness and prepare our children for the night. 1 and 2 and 3 and 4, these are not labels, these are geography, community, family. But if all of this is the purview of literacy instruction, we are going to need a little more light to work with. We will have to come out from behind the ELA so that we can see just how much distance we’re expected to cover in 90 minutes.
The English Language Arts exam tests the art of living in the modern world. You can disagree with me, but the fact that you are here to do so speaks to my point. Would you trade your ability to read and disagree with me for a million dollars? No? Well, you’re a 4, you’ve got it made. You are wealthy in the currency of the language of power. If it is my job to transmit this wealth to my poor students, I’m going to need a lot more to invest in than the image of the ELA exam. I’ll trust the ELA to tell me when my students are 4s, but I don’t trust it to tell me how to get them there.
And yet the ELA has spoken to me with the intimacy of nerves and neurons about what should not be done. We should not treat our 1s and 2s as though they need to earn 3s and 4s on the ELA. We should treat them like we treat 3s and 4s. But 3s and 4s don’t come from English classrooms alone. Where they go to school, English classrooms must justify themselves despite the test. The mission of such schools and such classrooms is to pamper the intellects of their 3s and 4s, an assignment that calls for more than the gruel of remedial instruction. Our poor 1s and 2s live on nonfiction passages and 3-Step Methods, main ideas and processes of elimination, explicit instruction and leveled libraries.
The ELA exam accurately reflects the student’s general level of reading comprehension. If we don’t like what we see, I don’t understand why we keep staring into the mirror.
3 comments:
Mr. Stickfigure says:
"The English Language Arts exam tests the art of living in the modern world...
If it is my job to transmit this wealth [currency of the language of power] to my poor students, I’m going to need a lot more to invest in than the image of the ELA exam. I’ll trust the ELA to tell me when my students are 4s, but I don’t trust it to tell me how to get them there.
And yet the ELA has spoken to me with the intimacy of nerves and neurons about what should not be done. We should not treat our 1s and 2s as though they need to earn 3s and 4s on the ELA. We should treat them like we treat 3s and 4s..."
I'll quibble a bit with Mr. Stickfigure here. I'd like to think that teachers should treat their 1s and 2s as though they COULD earn 3s or 4s. That is, in a very real sense, we can get what we want from-- and, more importantly, for-- our students if we ask for it.
But what should we ask for?
That's the rub, isn't it? The exam doesn's tell anyone what instruction should look like; rather, it's meant to be a trailing indicator of the effectiveness of the teachers' instruction.
I'm not very familiar with New York State's assessment system. (Well, actually, I'm not at all familiar with it...) However, the scoring standards for the exams must be based on some frameworks. If they're done well, they should provide the answer Mr. Stickman is searching for about the appearance of his instruction.
This brings us to another issue, one raised by Mr. Stickfigure in another of his posts: Too many people worry too much about preparing their students for the exams. That's understandable, but they're missinhg a critical point.
If the exams are well-designed trailing indicators of the effectiveness of instruction, and if the frameworks that undergird the exams are well done, then there's no reason to worry about the exams per se. That is, if our instruction is aligned to the frameworks and effectively delivered, then the exams will take care of themselves.
So, teachers should quit trying to game the system. In fact, they should quit trying to explicitly prepare students for the exams at all. Rather, they should worry about teaching their subjects well-- to every student.
That's the way to treat the 1s and 2s that Mr. Stickfigure is rightfully concerned about.
Thank you, Mr. Loyd, for crystallizing something irked me in a nebulous sense for some time. Rather than describe that galactic whirlwind, I'll quote the aphorism I will henceforth be relating to my colleagues on a daily basis--both with and without just attribution, I'm sure, but always with gratitude:
"So, teachers should quit trying to game the system. In fact, they should quit trying to explicitly prepare students for the exams at all. Rather, they should worry about teaching their subjects well-- to every student.
"That's the way to treat the 1s and 2s that Mr. Stickfigure is rightfully concerned about."
Thanks, also, for the immensely useful term, "trailing indicator." That, I trust,is an industry phrase, and one I wish I had intercepted two years ago.
The ELA is, indeed, a trailing indicator. I trust it is some indication of the upsidedownness of our ways that such a handy term has been so long out of my grasp. Even if I myself were inclined to simplicity, this pedagogical profusion would be enough to confuse and ensnare me.
It simply is a fact that, within the educational jurisdiction of which I am a part, the ELA leads instruction by the nose. By instinct, everyone knows this is wrong, but it is the same instinct that leads us to the illogical extreme of which I am so often guilty: that of hating the test.
Even a poorly designed, institutionally and culturally biased test is SOME sort of trailing indicator. As such, it can never be more than indication, after the fact, of the effectiveness of instruction. The ELA is a sophisticated test, perhaps too sophisticated, but probably to overshoot is the best way to miss the mark.
And yet, the devil lingers in me. When an educational jurisdiction reads its indicators and finds that the majority of its students are not hitting the mark, what are they supposed to do?
What they actually do do--in terms of this deleterious "test sophistication"--is precisely the path of least resistance, which is to say, the path that makes the most sense.
No one, after all, can tell us there's a better way. That is to say, the vast majority of those whose trailing indicators indicate academic success have been able to put the tests in the proper perspective from the get. In those schools, when the original exam indicated the previous year's instructional efficacy, it found it to be good. In order to maintain efficacy, nothing had to change. Such a school can point to nothing that it does without revealing three things we lack. This is the math of 1s and 2s and 3s and 4s, and when you add it all up it takes a lot more than what can be done inside the walls of any school to systematically and statistically produce 3s and 4s.
We are agreed, however, that chasing the tail of the ELA never amounted to more than dizziness and occasional nausea.
Mr. Stickfigure says:
"In those schools [with high test scores], when the original exam indicated the previous year's instructional efficacy, it found it to be good. In order to maintain efficacy, nothing had to change. Such a school can point to nothing that it does without revealing three [Does Mr. Stickfigure mean "the"?] things we lack. This is the math of 1s and 2s and 3s and 4s, and when you add it all up it takes a lot more than what can be done inside the walls of any school to systematically and statistically produce 3s and 4s."
It most certainly does take more than the schools can provide to effect real change in education or in the broader society.
That's why No Child Left Behind is misguided. It measures all schools against a single standard. Though decades of research demonstrate that parental income is the single most important variable affecting student test scores, NCLB demands that all schools, regardless of socioeconomic factors, meet the same standards. This is patently absurd; indeed, it is evil, in that it mandates failure for many schools.
However, the various professional organizations have reacted by demanding that standardized testing cease or at least be radically deemphasized. Given the public's attitude toward government of all kind and toward the schools in particular, that's not going to happen.
What should happen? The tests should be used to assess the performance of the schools and school districts, not the performance of the individual teachers and students. Each school and school district should be expected to improve over time, as measured against its own past performance. Schools that meet their individual targets should be recognized, and schools that don't should be the focus of targeted assistance.
This is to say that the trailing indicators should be used as such, not as predictors of future performance or, worse yet, as retention or graduation exams.
There's nothing wrong with standardized testing. After all, without some objective measure, how can anyone anywhere know whether any school is really doing a good job? The tests aren't the problem; rather, the way the testing data is used is the issue.
Post a Comment