Preview of Article:
A Test Scorer’s Lament
Illustrator: Stephen Kroninger
Having made my living since 1994 scoring standardized tests, I have but one question about No Child Left Behind: When the legislation talks of “scientifically-based research” — a phrase mentioned 110 times in the law — they’re not talking about the work I do, right? Not test scoring, right? I hope not, because if the 15 years I just spent in the business taught me anything, it’s that the scoring of tens of thousands of student responses to standardized tests per year is played out in the theatre of the absurd.
I’m not talking about the scoring of multiple-choice standardized tests; they’re scored electronically, but it’s not as foolproof as you’d like to believe. No, I’m talking about open-ended items and essay prompts where students fill up the blank pages of a test booklet with their own thoughts and words.
If your state has contracted that job out to one of the massive multinational, for-profit companies that largely make up the testing industry, you’ve surely been led to believe those student responses will be assessed as part of a systematic process as sophisticated as anything NASA could come up with. The testing companies probably told you all about the unassailable scoring rubrics they put together with the help of “national education experts” and “classroom teachers.” Perhaps they’ve told you how “qualified” and “well-trained” their “professional scorers” are. Certainly they’ve paraded around their omniscient “psychometricians,” those all-knowing statisticians with their inscrutable talk of “item response theory,” “T-values,” and the like. With all that grand jargon, it’s easy to assume that the job of scoring student responses involves bespectacled researchers in pristine, white lab coats meticulously reviewing each and every student response with scales, T-squares, microscopes, and magnifying glasses.
Well, think again.
