The Educational Costs of Standardization

Editor’s Note: The following article examines how high-stakes testing has affected teaching and learning in classrooms in Houston, the fifth-largest public school system in the United States.

Texas is the second-largest state, and its educational policies help set the national agenda. Furthermore, Texas has been cited, particularly by backers of presidential hopeful and Texas Governor George W. Bush, as a positive example of how high-stakes testing can act as a catalyst of education reform. Under the Texas Assessment of Academic Skills (TAAS), students cannot graduate if they fail the TAAS exams. Further, a principal’s pay is tied to the school’s performance on TAAS.

The following article provides an overview of how an emphasis on high-stakes testing affects teaching and learning in the classroom – particularly in schools with large percentages of African-American and Latino students who have traditionally scored lower on standardized tests than white students in more affluent areas.

The article is condensed from the final chapter of the forthcoming book Contradictions of School Reform: The Educational Costs of Standardized Testing. The book is based on research by Linda McNeil at a set of magnet schools. She visited the schools prior to the implementation of centralized accountability measures and then again after the reforms were imposed, documenting the effects on classroom practice.

. . . 

The town’s head librarian loved to encourage the children of his small, isolated farming community to read. He frequently went to the local school to read to the children. Most recently, he had been reading to a class of “at-risk” eighth-graders – students who had been held back two or more years in school. They loved his reading and his choices of books. He reports feeling very frustrated: the department chair has told him not to come any more to read to the students – they are too busy preparing for their TAAS test.

– unsolicited correspondence

Three in a row? No, No, No!

[Three answers “b” in a row? No, No, No!]

– one of several cheers taught to students at their daily pep rallies on test-taking strategies for the TAAS test

In many urban schools, particularly those whose students are predominantly poor and minority, the TAAS system of testing reduces both the quality of what is taught and the quantity of what is taught, because commercial test-prep materials are substituted for the regular curriculum. Reading skills, writing, and math are currently the subjects being tested by TAAS. Because the principal’s pay (and job contract) and the school’s reputation depend on the school’s scores, in those schools where students have traditionally not tested well on standardized tests, the regular curriculum in these subjects is frequently set aside in order that students can prepare for the test.

Common sense would suggest that if a teacher followed a traditional curriculum, even using the state’s textbook, the teaching of regular lessons would be preparation for success on the test. If students were able to do math problems, explain math concepts, and apply math skills in the regular sequence of lessons, then it should follow that they would do well on the test.

The tests, however, are not necessarily consistent with traditional teaching and learning. First, they are multiple-choice; they call for selecting among given answers. Second, they call for accurately darkening a circle beside the selected answer, without making stray marks on the paper.

In minority schools, in the urban school district where the magnet schools are located, and in many schools across the state, substantial class time is spent practicing bubbling in answers and learning to recognize “distractor” (obviously incorrect) answers. Students are drilled on such strategies as the one in the pep rally cheer quoted above: if you see you have answered “b” three times in a row, you know (“no, no, no”) that at least one of those answers is likely to be wrong, because the maker of a test would not be likely to construct three questions in a row with the same answer-indicator. (The basis for such advice comes from the publishers of test-prep materials, many of whom send consultants into schools – for a substantial price – to help plan pep rallies, to “train” teachers to use the TAAS-prep kits, and to ease the substitution of their TAAS-prep materials for the curriculum in classrooms where teachers stubbornly resist.)

LESS LATITUDE

Teachers, even those who know their subjects and their students well, have much less latitude when their principals purchase test-prep materials to be used in lieu of the regular curriculum. The decision to use such materials forces teachers to set aside their own best knowledge of their subject in order to drill their students on information whose primary (and often sole) usefulness is its likely inclusion on the test. Examples of this splitting of personal and professional knowledge, and the requirement to do so, abound. A particular example reveals not only how test-prep “teaching” diminishes the role of the teacher but also how it distances course content from the cultures of the students.

One teacher, a graduate of an Ivy League college, with a master’s degree at another select college, had spent considerable time and money assembling a rich collection of historical and literary works of importance in Latino culture. She had sought titles especially related to the American Southwest for her classes at a Latino high school. Her building of a classroom resource collection was extremely important given the school’s lack of a library and its lean instructional budget. Her students responded to her initiative with a real enthusiasm to study and learn. She was dismayed to see, upon returning one day from lunch, that the books for her week’s lessons had been set aside. In the center of her desk was a stack of test-prep booklets with a teacher’s guide and a note saying “use these instead of your regular curriculum until after the TAAS.” The TAAS test date was three months away. (The prep materials bore the logo “Guerrilla TAAS,” as in making war on the TAAS test; the booklet covers were military-camouflage colors; the Guerrilla TAAS consultants came to the school in camouflage gear to do a TAAS pep rally for the students and faculty.) This teacher reported that her principal, a person dedicated to these students and to helping them pass the TAAS in order to graduate, had spent almost $20,000, virtually the entire instructional budget for the year, on these materials. The cost was merely one problem. Inside the booklets for “reading” were single-page activities, with brief reading selections followed by TAAS-type answer choices. These students, who had been analyzing the poetry of Gary Soto and exploring the generational themes in Bless Me òltima, had to set aside this intellectual work to spend more than half of every class period working through the TAAS-prep booklet.

The imposition throughout the entire school of TAAS prep as a substitute curriculum recast the role of teachers, making them into people needing outside consultants to tell them (and “pep them up for”) ways to raise test scores. That these commercial materials were imposed proscribed the capacity of the teacher to resist. They also made it difficult for teachers to make accommodations at the margins, to try to hold onto the more substantive curriculum and cultural connections the magnet teachers for the most part had managed under the proficiency system to do.

TEACHER CONSTRAINTS

The limiting of the role of the teacher in shaping or negotiating the course content and the means of assessment causes problems beyond deciding what to teach. When their students’ learning is represented by the narrow indicators of a test like the TAAS, the teachers lose the capacity to bring into the discussion of the school program their knowledge of what children are learning. Teachers in urban schools say that to raise questions about the TAAS and about artificial test prep is characterized as being against minority students’ chances to get high test scores. Or it is portrayed as “not being a team player.” The test scores generated by centralized, standardized tests like the TAAS, and by the test-prep materials which prepare them for those tests, are not reliable indicators of learning. It is here where the effects on low-performing students, particularly minority students, begin to skew the possibilities for their access to a richer education.

At this school and other minority high schools where TAAS prep is replacing the curriculum, teachers report that even though many more students are passing TAAS “reading,” few of their students are actually readers. Few of them can use reading for assignments in literature, science, or history classes; few of them choose to read; few of them can make meaning of literature or connect writing and discussing to reading. In schools where TAAS reading scores are going up, there is little or no will to address this gap. First, so much publicity surrounds the rising scores, and the principals’ and superintendents’ bonuses are contingent on that rise, that the problem of nonreaders is silenced. Second, with the problem silenced, there can be no leverage to add the resources, change the teaching, or invite discussion about the sources of the problem. In fact, the opposite occurs: the rise in scores is used to justify even more TAAS prep, even more pep rallies, even more substituting of test-based programs for the regular curriculum.

TAAS AND READING

Advocates of TAAS might argue that passing the reading skills section of TAAS is better than not being able to read at all. However, there is first of all no evidence that these students “cannot read at all.” Second, teachers are reporting that the kind of test prep frequently done to raise test scores may actually hamper students’ ability to learn to read for meaning outside the test setting. In fact, students report that in the drills on the TAAS reading section, they frequently mark answers without reading the sample of text. They merely match keywords in an answer choice with keywords in the text. The definition of “reading” as captured on the test ignores a broad and sophisticated research base on the teaching of reading and on children’s development as language learners. When teachers are able to draw on this professional knowledge base, it does not lead them to testing formats like TAAS for help with their children’s reading.

Elementary teachers have expressed the concern that extensive prep for the reading section of TAAS actually undermines children’s ability to read sustained passages. The prep materials in reading, again purchased by principals eager to protect their performance contract or perhaps to help children pass the test, feature brief passages. After reading a passage, students are to answer practice questions (“Which of the following is the main idea?” “Which of the following would not make a good title for this paragraph?” “Which of the following was described as ‘ … ‘?”). The selected passage is not something they will see again; it is not even linked to the subsequent practice passage.

Students who practice these reading exercises day after day for months (many principals have teachers begin TAAS prep in September and do not let them revert to the “regular” curriculum until after the TAAS test in March) show a decreased ability to read longer works. A sixth-grade teacher who had selected a fourth-grade Newbery Award book for her class, thinking all the students could read and understand it, found that after reading for a few minutes the students stopped. They were accustomed to reading very brief, disjointed passages; they had difficulty carrying over information from the first chapter to a later one. Discussions with other upper elementary and middle school teachers confirm that students accustomed to TAAS prep, rather than literature, maybe internalizing the format of reading skills tests but not the habits needed to read for meaning.

The teaching of “writing,” also a subject tested by TAAS, has been reduced in many schools to daily practice of the essay form being tested that year. A teacher who is African-American and always alert to good educational opportunities for her sons was very pleased that her second son would be able to have the same excellent fourth-grade teacher under whom her oldest son had thrived. She was not prepared for the TAAS-based transformation of the fourth grade in the intervening years. She said that although the principal and teacher remained the same, the entire fourth-grade curriculum had been replaced by TAAS prep. Writing had become daily practice in “the persuasive essay,” consisting of five five-sentence paragraphs, a form which clearly qualifies as “school knowledge” in the most limited sense. What students had to say in these essays was of virtually no importance; conforming to the form was the requirement, and the students practiced every day. This mother knew that in Anglo schools, while there was some abuse of teaching through TAAS prep, most of the children were nevertheless learning to tailor their writing to their subjects, write in different voices and formats to different audiences, write to stretch their vocabularies.

THE FIVE-PARAGRAPH ESSAY

A principal of a middle- to upper-middle-class elementary school explained to an audience at a school reform conference that her teachers had heard that teachers at other schools were having their students practice the five-paragraph essay every day. They were concerned to hear that it had become the only form of writing done that year in their school. This principal, under much less pressure to contrive passing rates for her students on the TAAS, worked with her teachers to include the TAAS as one of many “audiences” when they teach students to develop voice and a sense of audience in their writing. Similarly, several high school teachers have told of discussions they had with their students about the TAAS writing exam. After learning more about TAAS, the students decided to think of the audience for their TAAS writing test as “bureaucrats sitting in little offices, waiting to count sentences and paragraphs.” These teachers, usually in high-performing schools and therefore not required to do TAAS prep, are in a similar way trying to make the test the subject of critical inquiry. This is not typical in low-performing schools where teachers and principals are using pep rallies and incentive prizes to get students to “buy-in” to these forms of evaluation.

The younger children growing up with TAAS prep may not always know (unless they compare with friends in private schools or have an older sibling whose learning was more substantive) how TAAS-prep reading and writing differ from good instruction. Older children, however, are not without skepticism that this system of testing is altering what they and their teachers jointly regard as important learning. Elaine, an eighth-grader, knows firsthand the artificiality of “TAAS writing.” In a previous grade, she won the citywide short-story writing award conferred by the local chapter of the National Council of Teachers of English. The next spring she received notice that she failed to pass the eighth-grade writing section of the TAAS because she “failed to provide sufficient supporting detail.” Elaine and her teacher both know that she is known in her school as a writer. What distinguishes her writing is its rich detail. They could speculate that perhaps the scanning of her TAAS writing missed, by its haste or its rigid format, the elaborative and “supporting” detail that characterizes her writing. The TAAS, and not the quality of her writing nor the English teachers’ judgment, lost credibility for her and for her parents as an indicator of her writing skills.

An eighth-grade class in a predominantly poor, Latino middle school demonstrated pointedly the intellectual subtraction resulting from the TAAS system of testing when the emphasis is on raising minority scores. In mid-September, a group of community visitors stepped into Mr. Sanchez’s class just as he was covering the blackboard with rules for semicolon usage. Using semicolons in writing seemed a useful and worthy lesson for eighth-graders working on their writing, so at first, the visitors watched without comment. While the students were copying the semicolon rules, the teacher explained: “We are having to do grammar until after the TAAS. I’m so excited – this year we have a whole nine weeks after the TAAS to do eighth-grade English. I always do Shakespeare with my students. And I have many stories that they love to read. Last year we didn’t have much time, but this year I will have a whole nine weeks.” The visitors were just then realizing the import of his words: he was to do TAAS prep from September until March, and then “teach eighth-grade English” only in the remaining nine weeks. And the teacher was made to feel grateful for all nine of those weeks. He explained that it was the will of the principal that they get the scores up and that everyone in the school was feeling the pressure. He knew that by focusing on the TAAS alone, his students would be getting far less than the eighth-grade curriculum studied by students in schools where the student demographics (middle class, predominantly white) would carry the scores, and they would be learning even less than his own students in the years before TAAS.

TAAS AND MATH

Under the TAAS-prep system, the teaching of mathematics, the third subject currently tested, is also highly truncated. TAAS tests math by having students choose among four or five possible answers. They are not asked to explain their answers, so if students have alternative ways of working a problem, their reasoning is not made visible on the test. Nor are their reasons for selecting “correct” answers. Being able to conceptualize in mathematics, being able to envision a solution and select among possible approaches, being able to articulate the reasoning behind the answer – none of these is tested by TAAS. TAAS tests computational accuracy and familiarity with basic operations. The reductive mathematics on the test is not adequate preparation for courses in more advanced mathematics. The TAAS-prep booklets, which emphasize test-taking strategies over mathematical reasoning, again create a gap between the content learned by poor and minority students in schools investing in TAAS-prep kits and the students in well-provisioned schools. In these latter schools, principals assume students will pass because of their family background and their having attended “good” schools in lower grades. They, therefore, support the teaching of the regular academic curriculum without substantial risk that to do so might “lower” the TAAS scores.

If a teacher wanted to avoid TAAS prep and focus on the students and the curriculum, then it would seem that the answer would be to teach a subject not yet tested by TAAS. At the Pathfinder school, Ms. Bartlett had claimed a space for teaching complex biology topics by shifting some of her teaching out from under the controls of the proficiency system (the predecessor of TAAS). She created elective courses and independent study seminars around such units of study as ecology and habitats (enabling her to integrate concepts and topics that were fragmented and sequenced separately under the proficiencies). She taught a biochemistry elective (using her knowledge gained from the medical school mentorship and crossing traditional subject boundaries) and, in some semesters, marine biology. Under the TAAS system of testing, teachers report that there are fewer and fewer venues in which they can do authentic teaching, even though officially only three subjects – math, reading, and writing – are tested. In poor and minority schools, especially, teaching untested subjects such as art, science, or social studies is not exempt from the pressures of TAAS prep. An art teacher with a reputation for engaging her Latino students in serious studio work, and for exciting students about being in school, was required to suspend the teaching of art in order to drill her students daily in TAAS grammar. By the time the grammar drills were completed, there was no time to set up for art projects. Her students were doubly losing: their treatment of grammar was artificial, aimed at correctness within the multiple-choice format of the test, rather than at fluency in their own writing; and they were denied an opportunity to develop their sense of color and design in art.

DISTORTING TEACHING

A history teacher in an under-resourced Latino high school worked with his colleagues to create a history curriculum that would maintain authentic content and yet incorporate some of the skills their students would need to do well on the TAAS. They included the writing of essays on historical topics and attention to reading skills. They had at first been given permission to create this on their own but later were told that they needed to set aside the teaching of history entirely in order to “cooperate with the rest of the faculty” in getting students to pass the TAAS. This history teacher’s assignment was to drill his students every day on math, a subject outside his field of expertise.

Science teachers who have spent a year in the Rice University Center for Education Model Science Lab (located in an urban middle school) updating their science knowledge and upgrading their capacity for laboratory-based teaching enter the program with the consent of their principals to implement what they have learned when they return to their schools. Many of these teachers have discovered on returning to their home schools that they are required, for as much as two to four months of the school year, to suspend the teaching of science in order to drill students on TAAS math. Again, their students in these urban schools are doubly penalized, first for losing out on the science that their peers in suburban schools are learning. Second, they are penalized by having to spend extra periods on low-level, disjointed math drills – math divorced from both the applications and conceptual understandings they will need if they are to hold their own later in upper-level math classes with middle-class students. It is unlikely that the middle-class students have been doing “math” from commercial test-prep booklets, rather than from math books, manipulatives, calculators, computers, and peer study groups. The TAAS, then, lowers the quality and quantity of even subjects not being tested in those schools where students have traditionally not tested well, the students who are poor and the minority.

From the losses in subjects being tested to the suspension of subjects not yet tested, it is clear that in any given year there may be weeks or even months of academic losses for these students whose principal’s bonus, or even the principal’s employment contract, depends on upward movement in student test scores. When the newspapers report “improvement” in the scores, the figures are taken at face value. There has been little public questioning into the dynamics producing these scores. The opportunity costs for minority students, who are in effect being used to ratchet up administrator pay, loom large. Those, too, were students of potential and promise, whose earlier schooling had not provided the background they should have going into secondary school but who, if taught a substantive curriculum, were able to develop into competent, even inspired, students. What is happening to and with the students under the test-prep system – and to curriculum content – is completely absent from public consideration under an accounting system that uses only one set of indicators on which to base administrative and economic decisions in schools.