Welcome to Measurement Inc.
In the heart of tobacco country, in a boxy brick building sandwiched between tire stores and a sprawling factory, you can find Ground Zero of New Jersey’s education reform movement.
And Texas’. And Maryland’s. And, in total, two dozen states’.
This is where essays from standardized tests come to be scored. On a single day in November, 70 paid graders working for the private firm Measurement Inc. scored 11,908 essays from New Jersey high school students. Sitting for seven hours a day in rooms that resemble large classrooms, they collectively pass judgment each year on the writing skills of millions of American students. They do not get rich at wages of $7.25 to $7.75 an hour, without benefits, but for many the flexibility is enough to offset the monotony.
But don’t expect to find teachers here. Ron Tanner is an ex-fighter pilot who has scored 150,000 essays in three years. Jeff Haubner is a recent college graduate who came to Durham in search of work. Karen Anderson scored papers as she was launching her art gallery.
All told, they’re a collection of college-educated jobbers who earn extra cash by evaluating the writing of the country’s youth. And because it’s here, and in six other scoring sites, where the essay portion of 4 million high-stakes standardized exams is graded, they are the ones who help determine who graduates, gets promoted, or is held back for remediation.
Surprised? Educators in New Jersey were. Not one of a dozen teachers or administrators interviewed, including several who managed testing programs, knew who scored the essays on these exams. All assumed that the graders were teachers or other professional educators.
Measurement Inc. says errors in grading are the exception and that virtually all papers are scored accurately by the part- time scorers. “I wondered … how professional people would be,” said Tanner, the former pilot. “But it’s incredible how seriously people take it. We’re not just scoring papers. We’re scoring people.”
GROWING USE OF ESSAYS
All but two states — Nebraska and Iowa — now use standardized tests to measure student achievement. For years, states required only that students shade in answer grids to multiple choice questions. Quick to grade and with one correct answer, they offered a fast snapshot of student achievement and were not subject to whims of individual graders.
But critics said these tests measure only memorization and test-taking skill. Essays, on the other hand, force students to reason and compose logical thoughts.
“Educators generally consider that [essays] tap more complex skills, more higher-ended skills,” said Gregory Cam- illi, an education professor at Rutgers University in New Jersey. “Educators also want these tests to send a message about what’s valued” by the state — in this case, writing.
Today, 42 states include essays on the standardized exams that they give to students in elementary, middle, or high school, according to the publication EducationWeek. That’s millions of essays each year. Somebody needs to grade them.
For some states, local teachers grade them. For others, Karen Anderson did.
Looking for extra money as she was launching her art studio, Anderson worked at Measurement in 1995, and she recalls the endless hours she spent pouring over student essays in Measurement’s Durham headquarters. “It was the most boring job I ever had,” she said.
“These papers are not that interest- ing,” she added. “To sit in a chair each day and read the same simplistic thoughts — it made you lose faith in the way people wrote.”
Yet Anderson took her job seriously, reading every paper from start to finish. “I never skipped over anything,” she said. And she had only praise for her col- leagues, some of whom were professionals with master’s degrees.
But there were others at Measurement who were less conscientious. Just ask Julian Harrison.
Harrison, a free-lance photographer who graded papers for three years until 1993, said he read 10,700 papers in one two-month period. “There were times I’d be reading a paper every 10 seconds,” he said. “It was horrific.”
At one point, Harrison would only briefly scan papers before issuing a grade, searching for clues such as a descriptive passage within a narrative to determine what grade to give. “You could skim them very quickly. You could read them very fast,” said Harrison, a freelance photographer. “You could actually — I know this sounds very bizarre — but you could put a number on these things with- out actually reading the paper.”
The same quality-control measures that the company used when Anderson read papers were there when Harrison graded essays, and they’re still used today. Still, like other readers, Harrison felt pressured to read his share of papers — and to get the $200 bonus that kicked in after 8,000 papers.
He knows he gave some papers the wrong grade.
“Either I read it too fast or I didn’t recognize what the child [meant] or may- be I got impatient because the child’s handwriting was very bad,” he said. “Maybe some of the readers weren’t careful enough, and maybe a child got a three instead of a four or a two instead of a four. Or maybe they failed.”
MEASUREMENT’S GRADING
Measurement is among a handful of firms that develop and grade standardized tests and their essays, earning up- wards of $6 for each paper it scores. In the month before testing season — usually late Spring, occasionally in the Fall — the firm advertises in local newspapers and on radio for part-timers looking to earn extra cash.
Measurement will interview almost anyone with a college degree, but it then employs a sophisticated monitoring process that constantly scans graders’ work to ensure that scoring is consistent among graders. The graders score up to 45,000 papers each day during peak season in the Spring. Sitting two to a table, they grade an average of 150 papers in a seven-hour day, although some go over 200.
Is this the best way to score such important exams? Company officials say it is, and testing experts say the firm uses sound methods. Every paper is read at least twice.
But teachers ask if it makes sense to use part-timers working in a former headache powder factory, far away from students’ schools. “You would think people in education would be selected,” said Joan Pra Sisto, a guidance counselor in the city of Passaic, NJ. “We know what we’re looking for.”
Neha Rana also has her doubts.
The teenager took the writing test last October and was crushed when the results came back. “After they told me I failed, I started crying,” Neha said. “The school switched my classes. They put me down from A-level classes to C-level classes.”
There was one problem, though: Neha should have passed. In the rush to score the essays, graders mistakenly failed her and three classmates. She was pulled from a class where she wrote research papers and spent months in remedial classes. After Neha’s school district asked Measurement to re-check her score, the firm caught its error.
HIGHSTAKES
Under the gun to produce high scores, it’s not just Neha who knows the importance of these tests. Superintendents may lose their jobs and trustees may lose elections, if schools don’t measure up.
So schools have responded. A principal in Ridgewood, NJ, a tony town with one of the top schools in the state, said his middle school now focuses less on in-depth writing and more on the quick essays demanded on the tests. Passaic is not the only district to banish failing students to remedial classes, and teachers in Paterson, NJ, a struggling urban district, say they’re less likely to teach poetry and creative writing so they can concentrate on the exam.
In fact, in what some fear will actually lead to a dumbing down of writing, many New Jersey schools now emphasize a five-paragraph essay format that they believe satisfies test graders — an introduction, three supporting paragraphs, and a conclusion. Paterson educators say that’s how they got a surge in writing scores, and other schools are using the same method.
“We’re basically making sure that our students write an essay in the format test graders want,” said James Bender, the superintendent in Little Falls, NJ.
